A realtime voice agent that listens, remembers and acts
A full speech-to-speech agent — realtime transcription, streamed responses, synthesized voice and tool calls — built to hold a real conversation, not play back a script.
Challenge
Most "AI voice" demos are a script wired to text-to-speech: no memory, no interruption handling, no ability to actually do anything. We wanted to prove a voice agent could hold a real conversation and complete real tasks inside it.
What we built
We built a full speech-to-speech pipeline: realtime speech-to-text, streamed LLM responses, text-to-speech, and voice-activity detection that handles interruptions mid-sentence. Conversational memory persists across turns, and the agent calls tools directly — looking up user information and booking consultations — through LangChain and LangGraph over WebSockets.
Results
- —Handles natural interruptions (barge-in) without losing conversational state.
- —Calls real tools mid-conversation — lookups and bookings — not just canned responses.
- —Full pipeline (speech-to-text → LLM → speech synthesis) runs over WebSockets with streamed, low-latency responses.