Skip to content

Connect: OpenAI Realtime and ElevenLabs Agents

Recipe · through our public MCP server and HTTP server · their docs checked 2026-11-16

Both voice platforms call remote MCP servers from their own cloud, so our server must be public, with auth. Read-only tools go over MCP. A tool with a side effect never runs on the platform's say-so: it becomes one of our approvals, and a person decides it.

sequenceDiagram
  participant O as OpenAI Realtime
  participant B as Your backend (sideband socket)
  participant S as public_app
  participant R as Reviewer
  O->>B: conversation.item.done (mcp_approval_request)
  B->>S: POST /tools/send_invoice/invoke (Idempotency-Key = item id)
  S-->>B: 202 pending_approval, apr_…
  B->>O: mcp_approval_response approve=false, reason "sent for review"
  R->>S: POST /approvals/apr_…/decide → runs once, returns result
  B->>O: system message with the result + response.create

Our side

def public_app(tools: Registry, host: str, agent_token: str, reviewer_token: str) -> Any:
    """REST at /, MCP at /mcp, one bearer check for both: for a host on the internet."""
    from mcp.server.transport_security import TransportSecuritySettings

    from agent_adapter.server import create_app

    tokens = {
        agent_token: Principal("hosted-agent"),  # the token the remote framework sends
        reviewer_token: Principal("reviewer@example.com", scopes=frozenset({APPROVER_SCOPE})),
    }
    security = TransportSecuritySettings(allowed_hosts=[host], allowed_origins=[f"https://{host}"])
    return create_app(
        tools,
        authenticate=bearer_auth(tokens),
        host="0.0.0.0",  # a public bind: the public-host checks apply
        allowed_hosts=[host],  # REST: any other Host header gets 421
        mcp_options={"transport_security": security},  # MCP: the same rule
    )


hosted = registry.subset(tags=["hosted"])  # no confirm tools: hosted runtimes cannot resume them

For OpenAI Realtime serve public_app(registry, …): REST needs the confirm tools to park them. For ElevenLabs, serve public_app(hosted, …).

Their side: OpenAI Realtime

Declare our server in session.update (or per turn in response.create):

{
  "type": "mcp",
  "server_label": "shop_tools",
  "server_url": "https://tools.example.com/mcp",
  "headers": { "Authorization": "Bearer <AA_AGENT_TOKEN>" },
  "allowed_tools": ["shipping_rate", "send_invoice"],
  "require_approval": { "never": { "tool_names": ["shipping_rate"] } }
}

Read-only calls run at once and arrive as mcp_call items. For send_invoice OpenAI sends an mcp_approval_request item instead (id, name, arguments, server_label). Your backend, on the sideband socket, maps it to our approval:

async def park_realtime_approval(
    item: dict[str, Any], rest: Any, session_id: str
) -> tuple[str, dict[str, Any]]:
    """OpenAI's `mcp_approval_request` -> one of our approvals; OpenAI never runs the call."""
    invoked = await rest.post(  # `rest`: an httpx client with base_url and the agent token
        f"/tools/{item['name']}/invoke",
        json={"arguments": json.loads(item["arguments"]), "run_id": session_id},
        headers={"Idempotency-Key": item["id"]},  # a replayed event parks nothing new
    )
    approval_id = invoked.json()["approval_id"]
    return approval_id, {  # send it on the Realtime socket: the model hears "sent for review"
        "type": "conversation.item.create",
        "item": {
            "type": "mcp_approval_response",
            "approval_request_id": item["id"],
            "approve": False,
            "reason": f"sent to a human reviewer as {approval_id}; it runs once they approve",
        },
    }


def realtime_outcome(approval_id: str, decided: dict[str, Any]) -> list[dict[str, Any]]:
    """The reviewer's decision (the body `/approvals/{id}/decide` returns), for the model."""
    said = f"Approval {approval_id}: {decided['status']}, result {decided.get('result')}"
    message = {
        "type": "message",
        "role": "system",
        "content": [{"type": "input_text", "text": said}],
    }
    return [{"type": "conversation.item.create", "item": message}, {"type": "response.create"}]
Step Code
conversation.item.done with item.type == "mcp_approval_request" approval_id, reply = await park_realtime_approval(item, rest, session_id), send reply
the reviewer decides your reviewer screen posts /approvals/{approval_id}/decide; the call runs once, here
back to the caller send each event of realtime_outcome(approval_id, decided) on the socket

approve: false keeps OpenAI from calling the tool itself. Had it said true, our server would still park the call over MCP, as no resume keys come back: nothing runs either way. The Idempotency-Key is the item id, so a replayed event parks no second approval. Use headers or authorization, never both.

The other route needs no mapping: declare send_invoice as a Realtime function tool, call /tools/send_invoice/invoke on function_call, and answer with function_call_output once decide returns.

Their side: ElevenLabs Agents

Turn on MCP for the workspace, then add a custom MCP server (dashboard, or the SDK):

server = elevenlabs.conversational_ai.mcp_servers.create(
    config={
        "url": "https://tools.example.com/mcp",
        "name": "shop_tools",
        "description": "Shop lookups: shipping rates.",
        "approval_policy": "always_ask",
    }
)

Pick the streamable HTTP transport (ElevenLabs also speaks SSE, which our server does not). In the dashboard, put the bearer token in Secret Token (sent as the Authorization header) and pick an approval mode: Always Ask, Fine-Grained Tool Approval or No Approval. Serve hosted here: ElevenLabs' approval is its own gate and never approves our confirm tools. For a side effect, add a server (webhook) tool that posts to /tools/send_invoice/invoke: the agent gets pending_approval and tells the caller it is waiting for review, and the reviewer's decide runs it.

Checklist

Check Why
a platform's approval adds a gate, never ours a confirm tool runs only after our reviewer decides (framework approvals)
one token per platform, with no APPROVER_SCOPE it passes through a third party; the platform cannot approve its own calls
the backend, not the browser, holds the agent token a token in a client bundle leaks
MCP is off for ElevenLabs workspaces in Zero Retention or HIPAA mode use the webhook route there
everything on the public-host checklist

Sources