Deploy: Docker, Kubernetes and probes

Run the HTTP tool server (create_app) in a container. This page gives a tested Dockerfile, a Kubernetes Deployment with liveness and readiness probes, and the rules for state that must be shared between pods.

File	Does
examples/deploy/app.py	the ASGI app: bearer token from a Secret, Redis stores when REDIS_URL is set
examples/deploy/Dockerfile	slim image, non-root user, uvicorn with a graceful shutdown
examples/deploy/k8s.yaml	Deployment + Service: /healthz, /readyz, a preStop sleep

The test suite builds the app, calls both probes and checks the files agree (ports, probe paths, the grace period).

The app

"""The container's ASGI app: `uvicorn app:app`; shared state in Redis when REDIS_URL is set."""

import os
from typing import Any

from agent_adapter import RedisApprovalStore, Registry
from agent_adapter.server import Principal, RedisIdempotencyStore, bearer_auth, create_app

registry = Registry()


@registry.tool(effects="read_only", idempotent=True)
async def shipping_rate(region: str) -> float:
    """Shipping rate for a region, in percent of the order value."""
    return {"eu": 5.0, "us": 18.0}.get(region, 12.0)


def shared_state(redis_url: str | None) -> dict[str, Any]:
    """Approvals and Idempotency-Key replays shared by every pod (in memory without Redis)."""
    if not redis_url:
        return {}
    from redis.asyncio import Redis  # only the deployed image installs redis

    def client() -> Any:
        return Redis.from_url(redis_url)

    return {
        "approvals": RedisApprovalStore(client),
        "idempotency": RedisIdempotencyStore(client),
    }


def build_app(env: dict[str, str]) -> Any:
    """The app for one environment; the token comes from a Kubernetes Secret, never the image."""
    tokens = {env["TOOLS_TOKEN"]: Principal("service@example.com")}
    hosts = [host.strip() for host in env.get("ALLOWED_HOSTS", "").split(",") if host.strip()]
    return create_app(
        registry,
        authenticate=bearer_auth(tokens),
        allowed_hosts=hosts or None,
        mcp=False,
        **shared_state(env.get("REDIS_URL")),
    )


app = build_app(dict(os.environ)) if "TOOLS_TOKEN" in os.environ else None

Data enters through POST /tools/{name}/invoke and the approval routes. It leaves as JSON results. Approvals and Idempotency-Key replays go to Redis, so any pod can resume a run.

The image

# The HTTP tool server image: slim base, non-root user, graceful shutdown inside the pod's grace period.
FROM python:3.13-slim

ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
RUN pip install "agent-adapter[server]" "redis>=5" \
 && useradd --create-home --uid 10001 tools
WORKDIR /srv
COPY app.py .
USER 10001
EXPOSE 8000

# One process per container: scale with replicas, not workers. Never gunicorn --preload.
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", \
     "--timeout-graceful-shutdown", "30", "--no-server-header"]

• One uvicorn process per container. Scale with replicas, not workers.
• --timeout-graceful-shutdown 30 lets open requests finish after SIGTERM.

Probes

Route	Auth	200	503
GET /healthz	none	the process is alive	never
GET /readyz	none	ready for new requests	starting (an MCP app before its lifespan ran), or draining

/readyz turns 503 when the lifespan starts shutting down, or when you call app.state.readiness.drain() yourself (a SIGTERM handler, a parent app's lifespan). Other routes keep serving while it drains. Both probes obey allowed_hosts: the kubelet sends the pod IP as Host, so set a probe Host header that is on the list, or the probe gets 421.

# The tool server Deployment: liveness on /healthz, readiness on /readyz, a preStop drain.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tools
spec:
  replicas: 2
  selector:
    matchLabels: {app: tools}
  template:
    metadata:
      labels: {app: tools}
    spec:
      terminationGracePeriodSeconds: 45  # > preStop sleep (10) + graceful shutdown (30)
      securityContext:
        runAsNonRoot: true
      containers:
        - name: tools
          image: registry.example.com/tools:1.0.0
          ports:
            - containerPort: 8000
          env:
            - name: ALLOWED_HOSTS
              value: tools,tools.default.svc.cluster.local
            - name: REDIS_URL
              value: redis://redis:6379/0
            - name: TOOLS_TOKEN
              valueFrom:
                secretKeyRef: {name: tools-auth, key: token}
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8000
              httpHeaders: [{name: Host, value: tools}]
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /readyz
              port: 8000
              httpHeaders: [{name: Host, value: tools}]
            periodSeconds: 5
            failureThreshold: 1
          lifecycle:
            preStop:
              exec:
                command: ["sleep", "10"]
---
apiVersion: v1
kind: Service
metadata:
  name: tools
spec:
  selector: {app: tools}
  ports:
    - port: 80
      targetPort: 8000

sequenceDiagram
  participant K as Kubernetes
  participant P as pod (uvicorn)
  K->>P: preStop: sleep 10
  Note over K: the endpoint is removed from the Service
  K->>P: SIGTERM
  P->>P: stop accepting, finish open requests (≤ 30 s)
  P->>P: lifespan shutdown: /readyz -> 503
  Note over K,P: terminationGracePeriodSeconds 45 > 10 + 30

gunicorn and fork()

Setup	Rule
gunicorn -k uvicorn.workers.UvicornWorker	fine; build the executor at import, nothing loop-bound
gunicorn --preload with uvicorn workers	never: loops and clients made before fork() leak into every worker
a module-level LoopRunner	safe before fork(): its loop starts on the first run(), a child starts its own
LoopLocal values	a forked child forgets the parent's values and builds new ones

Per process vs shared

Each worker process has its own copy of anything in memory. With more than one replica, pass a shared backend for each row that matters to you.

State	Default (per process)	Shared choice
parked approvals	MemoryApprovalStore	RedisApprovalStore, SQLAlchemyApprovalStore, DynamoDBApprovalStore (Database stores); SQLiteApprovalStore only on one host
Idempotency-Key replays	MemoryIdempotencyStore	RedisIdempotencyStore
run events (SSE)	MemoryEventStream	your own EventStream; else pin a run to one pod
cost ledger	Ledger	a LedgerSink that writes to your database
run guards, rate limits, budgets	in the Executor	per process by design: size limits per replica
tool cache	MemoryCache	any CachePort over Redis
LoopLocal clients	per loop	not shareable: each process opens its own

Executor, Registry and Ledger refuse to be pickled (ToolUsageError). Build them in each process; never send them to another one.