Voice call features¶
Everything on this page plugs into VoiceSession and runs offline,
with no account or key. Most features are a CallObserver, passed in observers=:
from agent_adapter.voice import (
CallRecorder, IdlePolicy, NoiseGate, ThinkingSound, VoicemailDetector, VoiceSession,
end_call_tool,
)
session = VoiceSession(
transport, stt=stt, tts=tts, turns=turns,
filters=[NoiseGate()], # the caller's audio, before anything else hears it
observers=[
IdlePolicy(after_s=8),
CallRecorder("calls/today.wav"),
ThinkingSound.ticks(),
VoicemailDetector(message="Hi, this is the clinic. Please call us back."),
],
)
registry.add(end_call_tool(session)) # the agent may hang up after its goodbye
agent = VoiceAgent(model, executor, session.speaker, mute_during_tools=True)
| Feature | How | What it does |
|---|---|---|
| Greeting or notice | agent.say(text) |
speaks outside a model turn; the history keeps what was heard |
| A line the caller cannot cut | agent.say(text, interruptible=False) |
caller speech during it is dropped, not queued |
| No cut-ins while a tool runs | VoiceAgent(mute_during_tools=True) |
speech while a tool runs is dropped |
| "uh-huh" never cuts | built in (backchannels=) |
a backchannel while the agent talks resumes playback |
| Silent caller | IdlePolicy(after_s=8, hang_up_after=2) |
asks "Are you still there?", then says goodbye and hangs up |
| The agent hangs up | end_call_tool(session) |
ends the call once the goodbye has played |
| Hang up from your code | await session.hang_up(after_speech=True) |
uses the transport's own hang-up where it has one |
| Typed input | session.send_text(text) |
a chat box beside the call; handled like a spoken turn |
| Recording | CallRecorder(path) |
stereo WAV, caller left, agent right, lined up in time |
| Thinking sound | ThinkingSound.ticks() or .from_wav(path) |
plays quietly while the agent thinks; stops for speech |
| Voicemail | VoicemailDetector(message=...) |
waits for the beep, leaves the message, hangs up |
| Noise filter | filters=[NoiseGate()] |
turns steady background noise down (numpy) |
Voicemail on outbound calls¶
VoicemailDetector uses two offline signals in the first window_s (20 s) of a call:
- the greeting's words: "leave a message", "not available" and other
PHRASES, or your ownclassify(text)(a model call, say); - the beep: a steady tone of 400 to 2,500 Hz lasting 160 ms.
Once it hears a greeting, the machine's words never reach the agent. With message=, the
agent waits for the beep (or 4 s of quiet), speaks the message without cut-ins and hangs up.
With on_detect(session, reason) you decide; reason is "greeting" or "beep".
Checked with real speech, not a person: two Kokoro voices read 3 greetings and 4 replies a person might give. Moonshine's transcripts matched all 6 greetings and none of the 8 replies. The longest steady tone found in any speech was 0 ms, and test beeps from 440 to 2,000 Hz were all found. A real line adds noise and codecs: test with your carrier before relying on it.
The noise filter, measured¶
NoiseGate learns the steady noise in each frequency band and turns it down by up to 12 dB.
scripts/noise_check.py mixes noise into six Kokoro sentences and compares Moonshine's
word error with and without the filter (lower is better):
| Noise | Signal to noise | Without | With NoiseGate |
|---|---|---|---|
| Pink (hiss, fans) | 0 dB | 9.4% | 10.0% |
| Pink | 5 dB | 5.1% | 3.8% |
| Pink | 10 dB | 3.8% | 3.8% |
| Crowd babble (three voices) | 0 dB | 114.9% | 71.6% |
| Crowd babble | 5 dB | 28.3% | 10.4% |
| Crowd babble | 10 dB | 5.0% | 5.5% |
It helps most where Moonshine struggles, a loud crowd behind the caller, and changes little elsewhere: Moonshine already copes with hiss. Above 100% means Moonshine wrote words that were never said. Six sentences is a small test, and a crowd that changes all the time is harder than ours, which repeats: try it on your own lines before turning it on.
Any function pcm -> pcm fits filters=, so a stronger denoiser plugs in the same way.
Recording¶
CallRecorder keeps the call in memory (up to max_minutes, 60) and writes the WAV when the
call ends. Each side is placed at the time it was heard. Tell callers they are recorded
where the law asks you to; the file holds their voice, so store it like other personal data.
Write your own¶
from agent_adapter.voice import CallObserver
class Transcript(CallObserver):
def __init__(self) -> None:
self.lines: list[str] = []
def caller_said(self, text: str) -> None:
self.lines.append(text) # return True instead to keep a line from the agent
started(session), caller_audio(pcm), agent_audio(pcm), caller_said(text) and
ended() are all optional. agent_audio can run on the sound card's thread: keep it quick.
An observer that raises is logged and the call goes on.