Live translation

A translation session interprets. Speech goes in and the same words come out in another language, a moment behind the speaker. The model never answers, greets or adds anything of its own. It uses the same connection and events as Nur, at wss://api.eesi.ai/v1/realtime/translations. Good uses:

A meeting in two languages. Leave the source language unset and each sentence is translated from whatever language it was spoken in.
Speaking to an audience. Use your cloned voice and people hear you, in their language.
Support across a language barrier, with no interpreter booked in advance.

For a recorded file, transcribe it, translate the text, and synthesize it instead.

Connect

Set the target language with language and, optionally, the source with source_language:

import asyncio, base64, json, os

import websockets

URL = (
    "wss://api.eesi.ai/v1/realtime/translations"
    "?model=nur-v1&language=es&source_language=en"
)


async def interpret(pcm_chunks, speaker):
    async with websockets.connect(
        URL, additional_headers={"Authorization": f"Bearer {os.environ['EESI_API_KEY']}"}
    ) as ws:
        async def send():
            for chunk in pcm_chunks:  # 16 kHz mono PCM16, about 100 ms each
                await ws.send(json.dumps({
                    "type": "input_audio_buffer.append",
                    "audio": base64.b64encode(chunk).decode(),
                }))
                await asyncio.sleep(0.1)  # real-time pace

        asyncio.create_task(send())
        async for raw in ws:
            event = json.loads(raw)
            if event["type"] == "response.output_audio_transcript.delta":
                print(event["delta"], end="", flush=True)
            elif event["type"] == "response.output_audio.delta":
                speaker.write(base64.b64decode(event["delta"]))

// Mint a ticket on your server first, as for a Nur session.
const ws = new WebSocket(
  "wss://api.eesi.ai/v1/realtime/translations" +
    `?model=nur-v1&language=es&token=${ticket}`,
);

Choose the languages

You can also set or change the languages on a running session:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "audio": {
      "output": { "language": "es" },
      "input": { "language": "en" }
    }
  }
}

Field	Meaning
`audio.output.language`	Target language, as an ISO 639-1 code. Required.
`audio.input.language`	Source language. Leave it out, or send `"auto"`, to detect it per sentence.

Put the target language in the same session.update that sets "type": "translation", or in the URL. Region codes such as pt-BR are fine. A sentence that is already in the target language produces no output.

Choose the voice

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "audio": { "output": { "language": "es", "voice": "ev_d116d96b" } }
  }
}

Any voice from your library works, including clones. A cloned voice is the version of this feature most people want: the listener hears the speaker, in their own language.

Text in or out

Audio and text work in both directions on the same session:

You send	You get
Audio (`input_audio_buffer.append`)	Translated speech and its text
Text (`conversation.item.create`, then `response.create`)	Translated speech and its text

Set "output_modalities": ["text"] for text only, such as live subtitles. It skips speech, so translations arrive sooner.

Tune it

Turn detection is set up for interpreting: a speaker starting a new sentence does not cut off the translation of the previous one. If you set audio.input.turn_detection yourself, keep "interrupt_response": false or the end of each translation gets clipped. Each speaker’s words are translated on their own. When someone else starts talking in the pause after a sentence, that sentence is translated then, without waiting for the next speaker to finish. One speaker pausing in the middle of their own sentence is still translated as one sentence. Anything in instructions refines the translation, such as register or terminology:

{
  "type": "session.update",
  "session": {
    "type": "translation",
    "instructions": "Use formal register. Keep product names in English.",
    "audio": { "output": { "language": "ja" } }
  }
}

Reconnecting

Translation sessions don’t resume. If the connection drops, open a new one and set the languages again.

Get started

Quickstarts

Nur

Speech

Audio intelligence

Live translation

Connect

Choose the languages

Choose the voice

Text in or out

Tune it

Reconnecting

​Connect

​Choose the languages

​Choose the voice

​Text in or out

​Tune it

​Reconnecting

Connect

Choose the languages

Choose the voice

Text in or out

Tune it

Reconnecting