Connect: OpenAI Realtime and ElevenLabs Agents¶
Recipe · through our public MCP server and HTTP server · their docs checked 2026-11-16
Both voice platforms call remote MCP servers from their own cloud, so our server must be public, with auth. Read-only tools go over MCP. A tool with a side effect never runs on the platform's say-so: it becomes one of our approvals, and a person decides it.
sequenceDiagram
participant O as OpenAI Realtime
participant B as Your backend (sideband socket)
participant S as public_app
participant R as Reviewer
O->>B: conversation.item.done (mcp_approval_request)
B->>S: POST /tools/send_invoice/invoke (Idempotency-Key = item id)
S-->>B: 202 pending_approval, apr_…
B->>O: mcp_approval_response approve=false, reason "sent for review"
R->>S: POST /approvals/apr_…/decide → runs once, returns result
B->>O: system message with the result + response.create
Our side¶
def public_app(tools: Registry, host: str, agent_token: str, reviewer_token: str) -> Any:
"""REST at /, MCP at /mcp, one bearer check for both: for a host on the internet."""
from mcp.server.transport_security import TransportSecuritySettings
from agent_adapter.server import create_app
tokens = {
agent_token: Principal("hosted-agent"), # the token the remote framework sends
reviewer_token: Principal("reviewer@example.com", scopes=frozenset({APPROVER_SCOPE})),
}
security = TransportSecuritySettings(allowed_hosts=[host], allowed_origins=[f"https://{host}"])
return create_app(
tools,
authenticate=bearer_auth(tokens),
host="0.0.0.0", # a public bind: the public-host checks apply
allowed_hosts=[host], # REST: any other Host header gets 421
mcp_options={"transport_security": security}, # MCP: the same rule
)
hosted = registry.subset(tags=["hosted"]) # no confirm tools: hosted runtimes cannot resume them
For OpenAI Realtime serve public_app(registry, …): REST needs the confirm tools to park
them. For ElevenLabs, serve public_app(hosted, …).
Their side: OpenAI Realtime¶
Declare our server in session.update (or per turn in response.create):
{
"type": "mcp",
"server_label": "shop_tools",
"server_url": "https://tools.example.com/mcp",
"headers": { "Authorization": "Bearer <AA_AGENT_TOKEN>" },
"allowed_tools": ["shipping_rate", "send_invoice"],
"require_approval": { "never": { "tool_names": ["shipping_rate"] } }
}
Read-only calls run at once and arrive as mcp_call items. For send_invoice OpenAI sends
an mcp_approval_request item instead (id, name, arguments, server_label). Your
backend, on the sideband socket, maps it to our approval:
async def park_realtime_approval(
item: dict[str, Any], rest: Any, session_id: str
) -> tuple[str, dict[str, Any]]:
"""OpenAI's `mcp_approval_request` -> one of our approvals; OpenAI never runs the call."""
invoked = await rest.post( # `rest`: an httpx client with base_url and the agent token
f"/tools/{item['name']}/invoke",
json={"arguments": json.loads(item["arguments"]), "run_id": session_id},
headers={"Idempotency-Key": item["id"]}, # a replayed event parks nothing new
)
approval_id = invoked.json()["approval_id"]
return approval_id, { # send it on the Realtime socket: the model hears "sent for review"
"type": "conversation.item.create",
"item": {
"type": "mcp_approval_response",
"approval_request_id": item["id"],
"approve": False,
"reason": f"sent to a human reviewer as {approval_id}; it runs once they approve",
},
}
def realtime_outcome(approval_id: str, decided: dict[str, Any]) -> list[dict[str, Any]]:
"""The reviewer's decision (the body `/approvals/{id}/decide` returns), for the model."""
said = f"Approval {approval_id}: {decided['status']}, result {decided.get('result')}"
message = {
"type": "message",
"role": "system",
"content": [{"type": "input_text", "text": said}],
}
return [{"type": "conversation.item.create", "item": message}, {"type": "response.create"}]
| Step | Code |
|---|---|
conversation.item.done with item.type == "mcp_approval_request" |
approval_id, reply = await park_realtime_approval(item, rest, session_id), send reply |
| the reviewer decides | your reviewer screen posts /approvals/{approval_id}/decide; the call runs once, here |
| back to the caller | send each event of realtime_outcome(approval_id, decided) on the socket |
approve: false keeps OpenAI from calling the tool itself. Had it said true, our server
would still park the call over MCP, as no resume keys come back: nothing runs either way.
The Idempotency-Key is the item id, so a replayed event parks no second approval. Use
headers or authorization, never both.
The other route needs no mapping: declare send_invoice as a Realtime function tool, call
/tools/send_invoice/invoke on function_call, and answer with function_call_output once
decide returns.
Their side: ElevenLabs Agents¶
Turn on MCP for the workspace, then add a custom MCP server (dashboard, or the SDK):
server = elevenlabs.conversational_ai.mcp_servers.create(
config={
"url": "https://tools.example.com/mcp",
"name": "shop_tools",
"description": "Shop lookups: shipping rates.",
"approval_policy": "always_ask",
}
)
Pick the streamable HTTP transport (ElevenLabs also speaks SSE, which our server does
not). In the dashboard, put the bearer token in Secret Token (sent as the
Authorization header) and pick an approval mode: Always Ask, Fine-Grained Tool Approval or No Approval.
Serve hosted here: ElevenLabs' approval is its own gate and never approves our confirm
tools. For a side effect, add a server (webhook) tool that posts to
/tools/send_invoice/invoke: the agent gets pending_approval and tells the caller it is
waiting for review, and the reviewer's decide runs it.
Checklist¶
| Check | Why |
|---|---|
| a platform's approval adds a gate, never ours | a confirm tool runs only after our reviewer decides (framework approvals) |
one token per platform, with no APPROVER_SCOPE |
it passes through a third party; the platform cannot approve its own calls |
| the backend, not the browser, holds the agent token | a token in a client bundle leaks |
| MCP is off for ElevenLabs workspaces in Zero Retention or HIPAA mode | use the webhook route there |
| everything on the public-host checklist |
Sources¶
- OpenAI Realtime with MCP: https://developers.openai.com/api/docs/guides/realtime-mcp
- ElevenLabs Agents MCP: https://elevenlabs.io/docs/eleven-agents/customization/tools/mcp
- ElevenLabs MCP security: https://elevenlabs.io/docs/eleven-agents/customization/tools/mcp/security