Tracing

Events, event sinks and OpenTelemetry spans. Concepts: Tracing.

Tracing: the event catalogue, event sinks and the OpenTelemetry sink.

REQUIRED_DATA = {EventType.RUN_STARTED: frozenset(), EventType.RUN_FINISHED: frozenset({'ok'}), EventType.TOOL_STARTED: frozenset({'tool', 'kind'}), EventType.TOOL_FINISHED: frozenset({'tool', 'latency_ms', 'attempts', 'cached'}), EventType.TOOL_FAILED: frozenset({'tool', 'error_kind', 'attempts', 'latency_ms'}), EventType.TOOL_RETRY: frozenset({'tool', 'attempt', 'delay_s', 'error_kind'}), EventType.TOOL_CACHED: frozenset({'tool'}), EventType.TOOL_PENDING: frozenset({'tool', 'approval_id'}), EventType.APPROVAL_REQUESTED: frozenset({'tool', 'approval_id'}), EventType.APPROVAL_DECIDED: frozenset({'tool', 'approval_id', 'approved', 'edited'}), EventType.APPROVAL_EXPIRED: frozenset({'tool', 'approval_id', 'outcome'}), EventType.LLM_STARTED: frozenset({*_LLM, 'stream'}), EventType.LLM_FINISHED: frozenset({*_LLM, 'latency_ms', 'usage', 'cost_usd', 'finish_reason'}), EventType.LLM_CACHED: frozenset(_LLM), EventType.LLM_FAILED: frozenset({*_LLM, 'error_kind'}), EventType.LLM_FALLBACK: frozenset({'from_provider', 'from_model', 'to_provider', 'to_model', 'error_kind'}), EventType.EMBEDDING_STARTED: frozenset(_EMBEDDING), EventType.EMBEDDING_FINISHED: frozenset({*_EMBEDDING, 'cache_hits', 'batches', 'latency_ms', 'usage', 'units', 'cost_usd'}), EventType.EMBEDDING_FAILED: frozenset({*_EMBEDDING, 'error_kind', 'latency_ms'}), EventType.RERANK_STARTED: frozenset(_RERANK), EventType.RERANK_FINISHED: frozenset({*_RERANK, 'latency_ms', 'usage', 'units', 'cost_usd'}), EventType.RERANK_FAILED: frozenset({*_RERANK, 'error_kind', 'latency_ms'}), EventType.FLOW_STARTED: frozenset({'flow'}), EventType.FLOW_FINISHED: frozenset({'flow', 'status', 'steps'}), EventType.NODE_STARTED: frozenset(_NODE), EventType.NODE_FINISHED: frozenset(_NODE), EventType.NODE_FAILED: frozenset({*_NODE, 'error_kind'}), EventType.NODE_PAUSED: frozenset({'flow', 'node', 'approval_id'}), EventType.NODE_CACHED: frozenset(_NODE), EventType.CHECKPOINT_SAVED: frozenset({'kind', 'step', 'status'}), EventType.RUN_RECOVERED: frozenset({'kind', 'step', 'replayed', 'rerun'}), EventType.BUDGET_WARNING: frozenset({'limit_usd', 'spent_usd'}), EventType.BUDGET_RESERVED: frozenset({'scope', 'limit_usd', 'spent_usd', 'reserved_usd'}), EventType.BUDGET_EXCEEDED: frozenset({'scope', 'limit_usd', 'spent_usd'}), EventType.CONTEXT_COMPRESSED: frozenset({*_CONTEXT, 'tokens_before', 'tokens_after', 'target_tokens', 'messages_before', 'messages_after', 'steps', 'summary_cost_usd'}), EventType.CONTEXT_COMPRESS_FAILED: frozenset({*_CONTEXT, 'error_kind'}), EventType.GUARDRAIL_REDACTED: frozenset({'guard', 'stage', 'count', 'kinds'}), EventType.MEMORY_ADDED: frozenset(_MEMORY), EventType.MEMORY_SEARCHED: frozenset({*_MEMORY, 'k', 'mode'}), EventType.MEMORY_DELETED: frozenset(_MEMORY), EventType.MEMORY_EXPORTED: frozenset(_MEMORY), EventType.RETRIEVAL_SEARCHED: frozenset({*_RETRIEVAL, 'count', 'ids', 'dropped'}), EventType.RETRIEVAL_FAILED: frozenset({*_RETRIEVAL, 'error_kind'}), EventType.SANDBOX_STARTED: frozenset({*_SANDBOX, 'code_bytes', 'files', 'network', 'timeout_s'}), EventType.SANDBOX_FINISHED: frozenset({*_SANDBOX, 'exit_code', 'timed_out', 'duration_ms', 'stdout_bytes', 'stderr_bytes', 'truncated', 'files'}), EventType.SANDBOX_FAILED: frozenset({*_SANDBOX, 'error_kind', 'duration_ms'})} module-attribute

CallbackEventSink

Calls fn(event); a failing callback is logged, never raised.

EventLog

Keeps the latest max_events events in memory, oldest dropped first.

of_type(prefix)

Events whose type starts with prefix, in emission order.

EventSink

Bases: Protocol

Receives trace events; emit is sync, non-blocking and never raises.

EventType

Event type strings; anything else must start with custom..

FanoutSink

Sends each event to every sink in order; a failing sink does not stop the others.

sinks property

The sinks each event goes to, in order.

LoggingSink

Logs each event at level; content keys only with capture_content and at DEBUG.

level property

The level events are logged at.

logger property

The logger events are written to.

OTelSink

An EventSink exporting GenAI-convention spans and metrics (needs opentelemetry-api).

capture_content property

Whether opt-in content attributes (messages, arguments, results) are exported.

__init__(*, tracer_provider=None, meter_provider=None, capture_content=False, instrumentation_name='agent_adapter')

Bind a tracer and meter (global providers when None).

Raises:

Type	Description
MissingDependency	opentelemetry-api is not installed (agent-adapter[otel]).

SinkHub

An executor's stable sink: base (the constructor's sink), then every added sink.

Contexts and models keep the hub itself, so add and remove reach them all. Members are a copy-on-write tuple: emit never locks.

base property

The constructor's sink (fixed for the hub's life), or None.

extra property

The sinks added after construction, in order.

sinks property

Every member, base first; empty when there is nothing to send to.

add(sink)

Add sink once; False when it is already a member.

remove(sink)

Remove an added sink; False when it was not added (the base stays).

emit(sink, event)

Send event to sink; a no-op for None. Sink failures are logged, never raised.

make_event(type, ctx=None, *, call_id=None, **data)

An event with ids from ctx; an explicit call_id makes ctx.call_id the parent.

Raises:

Type	Description
ToolUsageError	unknown type (not in the catalogue, not custom.*) or missing data keys.