The next generation of AI will not be defined by how well it generates speech, but by how well it understands and navigates interaction itself. From AI that responds to AI that participates.Three things make that possible, and you can see all three in the API. A representation of conversation that keeps what a transcript throws away. Simulation, where models talk to models until something breaks. And conversation-native models trained on continuous audio instead of on text scraped out of it.
Where interaction shows
These are the surfaces where the difference is audible. Start here, not with the file endpoints.Live
Speech in, speech out, one model, one socket. Barge-in, end-of-turn
detection and cancellation are handled by the server, not by you.
WS /v1/realtime.Duet
Two live sessions cross-piped into one conversation. This is how a persona
gets tested against a difficult counterpart before a customer meets it.
The rest of the stack
Everything a conversation needs around it, on the same key and the same account.Text to speech
Synthesis in a built-in or cloned voice.
POST /v1/audio/speech.Speech to text
Recorded audio to text, with timings.
POST /v1/audio/transcriptions.Voices
Clone from a few seconds of speech, or use the built-in library.
POST /v1/voices.Live translation
Interpretation as its own session type. You speak; the room hears you a
beat later in another language.
Start
Quickstart
A key, one synthesized line, and a live conversation. Ten minutes.
What EESI does today
The honest map: every capability, the model behind it, and where it stops.
Models
The catalog, what each id serves, and why you pin the version.
Authentication
Keys, the two header forms, and browser tickets for a WebSocket.
Two things to know before you build
They explain most of the API surface, and getting either wrong looks like a bad model rather than a mistake.Live audio is a different model, not the same one faster
Live audio is a different model, not the same one faster
The expensive mistake is assembling a conversation out of the file
endpoints: slice the microphone, POST each slice to transcription, send the
text to a chat model, synthesize the reply. That costs three round trips a
turn, and it destroys turn boundaries — fixed slices cut words in half and
tell the model nothing about whether a person has finished speaking.Audio you already have is a file. Audio arriving as someone speaks is a
conversation. See Realtime interaction.
Every voice resolves to a stored reference clip
Every voice resolves to a stored reference clip
Cloned or built-in, a voice is always a piece of reference audio. Without
one, the synthesis model samples a new voice per generation, and a call
changes speaker between sentences. This is why voices are a resource you
create rather than a string you pass. See Voices.