Realtime interaction

A file endpoint answers a question you have already finished asking. A live session has to decide what to do while you are still talking — whether that pause was the end of your sentence, whether the thing you started saying replaces the thing you were about to say, whether to answer at all yet. That decision is the product. Live and Duet are two shapes of it.

Live

Speech in, speech out, one model, one socket. The server owns end-of-turn detection and barge-in. WS /v1/realtime.

Duet

Two live sessions with each one’s output wired into the other’s input. A conversation you can listen to before a customer has one.

Pick the surface

Your situation	Use	Page
A person is speaking and expects an answer now	`WS /v1/realtime`	Live
A person is speaking and wants the same words in another language	`WS /v1/realtime/translations`	Live translation
You want to hear your agent handle a difficult counterpart	Two `WS /v1/realtime` sockets	Duet
You already have the audio file	`POST /v1/audio/transcriptions`	Speech to text
You have the text and want it spoken	`POST /v1/audio/speech`	Text to speech

The last two rows are the important ones. A file endpoint is cheaper, needs no socket, and can be retried. If nobody is waiting, use one.

Chunking a microphone into repeated POST /v1/audio/transcriptions calls is not a conversation. Each request is decoded on its own, so turn boundaries, overlap and timing — the things a live session exists to get right — are destroyed before the model sees them.

What every live session shares

Live, translation and each half of a duet are the same kind of session on the wire and in your account. Anything you build for one works for the others. One socket, both directions. Audio and transcripts of both speakers arrive on the same connection as the reply. There is no second service to run for captions. The gateway sets the base instructions. Every session.update you send is composed against a platform prompt before it reaches the model, and your instructions are appended after it, where trailing text carries the most weight. This is why interpretation is its own session type rather than a prompt: the conversational persona tells the model to introduce itself and not to translate unasked, and it wins. A record is left behind. Every session opens a row with its model, source, duration and, for a duet, the id of its other half — so a finished conversation can always be found. The transcript and the recording are a deployment decision: an environment that does not retain session audio keeps neither, and the row’s per-turn timing goes with them. Per-turn latency is aggregated organization-wide either way. See Sessions and Latency. It is metered per minute of wall clock, at the same rate as synthesis and transcription, starting when the upstream pipeline actually connects — a handshake that never came up is not charged. The balance and the free-tier day are re-checked once a minute while the call is up, so a limit can end an established session. See Pricing and limits. It has a ceiling. Three hours of wall clock, and five minutes without a frame from the client. The idle timer watches your frames only, so an open microphone renews itself and a text-only session does not — see Connections.

Models resolve to backends

A public model id is a name for a serving backend, and a backend that is not configured in this environment is not listed and answers 404 model_not_found. The catalog therefore differs between deployments. Read it rather than hardcoding an id:

curl https://api.eesi.ai/v1/audio/models \
  -H "Authorization: Bearer $EESI_API_KEY"

from eesi_sdk import EESIClient

with EESIClient(api_key="sk-eesi-...") as client:
    # `kind` is an Enum, not a str — compare `.value`, or the filter is empty.
    realtime = [m for m in client.list_speech_models().data if m.kind.value == "realtime"]
    for model in realtime:
        print(model.id, model.supports_cloning)

import { EESIClient } from "@eesi/sdk";

const client = new EESIClient({ apiKey: process.env.EESI_API_KEY });
const { data } = await client.listSpeechModels();
console.log(data.filter((model) => model.kind === "realtime").map((m) => m.id));

One id serves realtime: nur-realtime-v1, which speaks in cloned voices. There were two — a full-duplex omni model sat beside it, unreachable in every shipped deployment because no environment ever filled its backend slot — and offering a choice nothing could serve was worse than offering none. Models has the capability flags.

Capacity, and what a full pool looks like

Each live session holds a pipeline in the serving pool for its whole duration. There is no per-organization concurrency cap in front of that pool today: the controls on a caller are authentication, the credit balance, the free-tier day, the per-minute request budget, and the session ceilings above. When the pool has no slot, the server waits up to ten seconds for one that is already draining, then refuses. Your client sees an error event with error.type of session_limit_reached, then a close with code 1013. That is worth retrying with backoff. A close with 1011 is not — it means the backend is unreachable, and retrying changes nothing until it is back. Connections has every close code.

Where to go next

Live

Connect, configure the session, stream audio, play the reply.

Realtime events

Every event in both directions, and the two that read backwards.

Turn-taking and barge-in

How the server decides you stopped, and what your client owes an interruption.

Connections that survive

Tickets, keepalives, close codes, and resuming a dropped session.

Start here