Stream Message Layer - Sanas Developer Hub

The Stream message layer defines the application protocol shared by the WebSocket Stream API and the WebRTC API. It covers session setup, pipeline configuration, turn finalization, transcription and translation events, lifecycle events, errors, and usage modes. Transport mapping:

API	JSON messages	Audio
WebSocket Stream API	WebSocket text frames	Raw PCM bytes in WebSocket binary frames
WebRTC API	Stringified JSON on the WebRTC data channel	WebRTC audio media tracks

Do not send audio in JSON messages. The init, configure, flush, and server event shapes are the same across transports.

Protocol Overview

Connection Lifecycle

Key Concepts

Init: Sent once per connection. Sets session-level parameters (sample rates, conversation ID).
Configure: Sent after init and again at any point to change languages, voice, or glossary. Each configure starts a new pipeline configuration. The server responds with configured when ready.
Flush: Signals the server to finalize the current input and produce all remaining output. The server responds with flushed when complete. Use this for manual turn-taking or to force processing of buffered audio.
Utterance: A continuous segment of speech. The server assigns an incrementing utterance_idx to each detected utterance. When VAD (Voice Activity Detection) is active, the server automatically splits speech into utterances.

Client Messages (Client -> Server)

Client control messages are JSON objects with a "type" field. Send them as WebSocket text frames on the WebSocket transport, or as stringified JSON messages on the WebRTC data channel. Audio transport is described in Input Audio.

`init`

Required as the first message on every connection. Sets connection-level parameters that remain fixed for the session lifetime.

{
  "type": "init",
  "conversation_id": "optional-id",
  "session_name": "my-session",
  "input_sample_rate": 16000,
  "output_sample_rate": 16000,
  "realtime_playback": true
}

Field	Type	Required	Default	Description
`type`	string	Yes	—	Must be `"init"`
`conversation_id`	string	No	auto-generated UUID	Identifier to group related sessions (e.g., a multi-turn conversation). If empty or omitted, the server auto-generates a UUID.
`session_name`	string	No	`""`	Human-readable label for the session
`input_sample_rate`	integer	No	`16000`	Sample rate of input audio in Hz. Allowed: `16000`
`output_sample_rate`	integer	No	`16000`	Desired sample rate of output audio in Hz. Allowed: `16000`
`realtime_playback`	boolean	No	`false`	Set to `true` if you intend to play output audio in real-time as it streams. This enables server-side pacing optimizations for simultaneous mode. Set to `false` for consecutive mode where the full translation audio is played after the speaker finishes.

`configure`

Initializes or reconfigures the translation pipeline. Must be sent after init. Can be sent again mid-session to change languages, voice, glossary, or features without reconnecting.

{
  "type": "configure",
  "language_routes": [
    {"lang_in": "en-US", "lang_out": "es-ES"}
  ],
  "features": [],
  "voice_id": null,
  "glossary": [
    {"terms": {"en-US": "Sanas", "es-ES": "Sanas"}}
  ],
  "request_id": "optional-correlation-id"
}

Field	Type	Required	Default	Description
`type`	string	Yes	—	Must be `"configure"`
`language_routes`	array	No	`[]`	Array of language route objects (see below)
`features`	array	No	`[]`	Pipeline features to enable (see Features)
`voice_id`	string or null	No	`null`	Custom voice ID for speech synthesis. If null, uses a default voice for the target language.
`glossary`	array	No	`[]`	Array of glossary entry objects (see below)
`request_id`	string or null	No	`null`	Client-specified ID echoed back in the `configured` response for correlation

The server responds with a configured message when the pipeline is ready to accept audio.

Language Routes

Language routes define the translation directions the pipeline supports. Each route specifies a source and target language.

{"lang_in": "en-US", "lang_out": "es-ES"}

Field	Type	Description
`lang_in`	string	Source language code (BCP-47, e.g., `"en-US"`, `"es-ES"`, `"fr-FR"`)
`lang_out`	string	Target language code

Single route — translates one direction only:

"language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"}
]

Bidirectional routes — the server detects which language is being spoken and translates to the other. Use this when both parties may speak either language (replaces the old can_lang_swap behavior):

"language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"},
  {"lang_in": "es-ES", "lang_out": "en-US"}
]

When multiple routes are provided, the server performs language identification on the incoming audio and selects the matching route. You will receive a language_route event indicating which route was chosen for each utterance. Auto-detect mode (any language to one target) — use "*" as lang_in to accept any supported language and translate it to a fixed target. The server identifies the spoken language automatically and translates to the specified output language:

"language_routes": [
  {"lang_in": "*", "lang_out": "en-US"}
]

This translates any incoming language to English. The server will send a language_route event with the detected language filled in:

{"type": "language_route", "utterance_idx": 0, "lang_in": "ja-JP", "lang_out": "en-US", "is_final": true}

In auto-detect modes, the server may send tentative language_route messages (is_final: false) while language identification is still refining, followed by a single is_final: true message once the route is locked in. Clients that only care about the final answer can ignore tentative messages and act on is_final: true. You can combine auto-detect with an explicit route. For example, to translate English to Spanish and auto-detect everything else to English:

"language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"},
  {"lang_in": "*", "lang_out": "en-US"}
]

NOTE: When multiple language routes are specified, they will be matched in list order. The first route to match wins. In the following language route configuration, the server will always choose the first route because * matches any detected language. In this case, the consequence is: English speech will remain as English.

"language_routes": [
  {"lang_in": "*", "lang_out": "en-US"}
  {"lang_in": "en-US", "lang_out": "es-ES"},
]

Wildcard catch-all route — add a "*" -> "*" route alongside your normal routes to handle speech in unexpected languages gracefully. When the detected language does not match any explicit route, the wildcard route is selected and the pipeline enters a transcription-only mode (no translation or TTS output). This is useful for prompting the user to switch languages:

"language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"},
  {"lang_in": "es-ES", "lang_out": "en-US"},
  {"lang_in": "*", "lang_out": "*"}
]

When the wildcard catch-all route is triggered, you will receive a language_route event with the detected language echoed into both lang_in and lang_out. For the configuration above, if the user speaks French you will see:

{"type": "language_route", "utterance_idx": 1, "lang_in": "fr-FR", "lang_out": "fr-FR", "is_final": true}

Note that the server then emits transcription / transcription_ended for the utterance but no translation, translation_ended, output_audio, output_speech_started, or output_speech_ended (transcription-only mode). You can use this language_route to show a prompt to the user, for example: “It seems like you are speaking French. Would you like to switch your language?” If the user confirms, send a new configure with the appropriate routes.

Glossary Entries

Glossary entries either (a) keep a phrase verbatim across languages or (b) lock in a specific translation between two languages. Each entry is a map of language codes to the desired term in that language. Keep a phrase verbatim across languages — use "*". For brand names, product names, and other proper nouns that should never be translated, use the "*" wildcard:

{"glossary": [
  {"terms": {"*": "Sanas"}},
  {"terms": {"*": "ACME"}}
]}

This tells the pipeline that “Sanas” should remain “Sanas” in every language, and “ACME” should remain “ACME” in every language. Lock in a specific translation between 2+ languages — use language codes. To force a phrase in one language to a specific phrase in another, list each phrase by its language code:

{
  "terms": {
    "es": "servicio al cliente",
    "fr": "service client",
    "en": "customer service"
  }
}

Here, “servicio al cliente” spoken in Spanish becomes “service client” when translating to French, and “service client” spoken in French becomes “servicio al cliente” when translating to Spanish. A glossary entry only takes effect on a language route when both the input and output languages are specified. If either side isn’t covered, the entry is ignored for that route. Note that using a short code like es will match all variants of Spanish (es-419 and es-ES). You are also free to specify a long code (es-419) if you need more fine-grained tuning. Combining both. You can mix "*" with explicit language codes in the same entry. The "*" then acts as the term used whenever a route’s input or output language isn’t explicitly listed. For example, {"terms": {"*": "customer service", "es": "servicio al cliente", "fr": "service client"}} resolves Spanish → French to “service client” via the explicit codes, while Spanish → Russian falls back to “customer service” via the wildcard.

Features

The features array controls which pipeline mode is used. By default, you can leave the features array empty or omitted. Note that feature names are case-sensitive:

Feature	Description
`"consecutive"`	By default, simultaneous mode is used, producing streaming translation and audio output in real-time as the speaker talks. This feature enables consecutive translation mode. The pipeline waits for the speaker to finish (via VAD or manual flush), then produces the full translation and synthesized audio.
`"detect_languages_once"`	By default, when `language_routes` actually require detection (any `"*"` `lang_in`, or multiple routes), every utterance redetects the language. This feature locks the session to only detect language a single time.
`"hide_output_text_boundary"`	Suppresses `output_text_boundary` emission. By default the server aligns TTS word timestamps back to the transcription/translation and emits boundary messages, suitable for word-level highlighting. Set this feature to skip all alignment work.
`"skip_logging"`	If not already, opts this session out of server-side recording. Once set on any `configure` in the session, it sticks for the lifetime of the stream.

Input Audio

Streams audio data to the server after receiving configured. Audio transport depends on the API you are using:

Transport	Input audio behavior
WebSocket Stream API	Send raw PCM bytes as Binary WebSocket frames. Do not wrap audio in JSON.
WebRTC API	Send microphone audio on the WebRTC audio media track. Do not send audio on the data channel.

For WebSocket binary audio, use this format:

Encoding: 16-bit signed integer PCM (little-endian)
Channels: Mono (1 channel)
Sample rate: Must match input_sample_rate from init

`flush`

Signals the server to finalize processing of all buffered audio and produce remaining results. The server will respond with a flushed message after all output has been sent.

{
  "type": "flush",
  "request_id": "optional-correlation-id"
}

Field	Type	Required	Default	Description
`type`	string	Yes	—	Must be `"flush"`
`request_id`	string or null	No	`null`	Client-specified ID echoed back in the `flushed` response for correlation

Server Messages (Server -> Client)

Server control and text messages are JSON objects with a "type" field. Receive them as WebSocket text frames on the WebSocket transport, or as stringified JSON messages on the WebRTC data channel. Audio transport is described in Output Audio. Note on optional fields: Fields documented as “or null” (such as request_id, probability, utterance_idx on errors) are omitted from the JSON when not present, rather than sent as "field": null. Your client should treat a missing key the same as a null value.

`configured`

Confirms the pipeline is ready to accept audio after a configure message.

{
  "type": "configured",
  "request_id": "r1"
}

Field	Type	Description
`type`	string	`"configured"`
`request_id`	string or null	Echoes the `request_id` from the corresponding `configure` message, if provided

`input_speech_started`

Indicates the server’s VAD has detected the start of speech in the input audio.

{
  "type": "input_speech_started",
  "utterance_idx": 0,
  "input_time": 0.5
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance that started
`input_time`	float	Timestamp in seconds (relative to the start of the input audio stream) when speech was detected

`language_route`

Indicates which language route was selected for the current utterance. Sent when the system identifies the spoken language, particularly useful when multiple language routes are configured.

{
  "type": "language_route",
  "utterance_idx": 0,
  "lang_in": "en-US",
  "lang_out": "es-ES",
  "is_final": true
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance this route applies to
`lang_in`	string	Detected/confirmed input language
`lang_out`	string	Target output language
`is_final`	boolean	`true` when this is the locked-in route for the utterance

`transcription`

Real-time transcription of the input audio. Complete words have been finalized and will not change. Partial words are in-progress and may be updated in subsequent messages.

{
  "type": "transcription",
  "utterance_idx": 0,
  "complete": [
    {"word": "Hello ", "start": 0.0, "end": 0.5, "probability": 0.98}
  ],
  "partial": [
    {"word": "world", "start": 0.5, "end": 0.8, "probability": 0.85}
  ]
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance this transcription belongs to
`complete`	Word[]	Finalized words — will not change and are sent only once. Store these on the client for display.
`partial`	Word[]	In-progress words — may be updated or replaced in subsequent messages

Word object:

Field	Type	Description
`word`	string	The transcribed word (may include trailing whitespace)
`start`	float	Start time in seconds relative to the beginning of the utterance’s input audio
`end`	float	End time in seconds
`probability`	float or null	Confidence score (0.0 to 1.0), or null if unavailable

`transcription_ended`

Indicates that all transcription for a given utterance is complete. No further transcription messages will be sent for this utterance.

{
  "type": "transcription_ended",
  "utterance_idx": 0
}

`translation`

Translated text derived from the transcription. Like transcription, includes complete (finalized) and partial (in-progress) words.

{
  "type": "translation",
  "utterance_idx": 0,
  "complete": [
    {"word": "Hola ", "start": 0.0, "end": 0.5, "probability": 0.95}
  ],
  "partial": [
    {"word": "mundo", "start": 0.5, "end": 0.8, "probability": 0.85}
  ]
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance this translation belongs to
`complete`	Word[]	Finalized translated words
`partial`	Word[]	In-progress translated words

`translation_ended`

Indicates that all translation for a given utterance is complete.

{
  "type": "translation_ended",
  "utterance_idx": 0
}

`input_speech_ended`

Indicates the server’s VAD has detected the end of speech in the input audio. After this, the server will not accept further input audio for this utterance and will finalize the translation.

{
  "type": "input_speech_ended",
  "utterance_idx": 0,
  "input_time": 3.2
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance that ended
`input_time`	float	Timestamp in seconds when end-of-speech was detected

`output_speech_started`

Indicates the server has begun sending synthesized output audio for an utterance.

{
  "type": "output_speech_started",
  "utterance_idx": 0,
  "output_time": 0.0
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance
`output_time`	float	Timestamp in the output audio stream (seconds)

Output Audio

Synthesized audio from the translated text. Audio transport depends on the API you are using:

Transport	Output audio behavior
WebSocket Stream API	Receive raw PCM bytes as Binary WebSocket frames. Audio is not wrapped in JSON.
WebRTC API	Receive translated audio on the WebRTC audio media track. Audio is not sent on the data channel.

For WebSocket binary audio, the format is:

Encoding: 16-bit signed integer PCM (little-endian)
Channels: Mono (1 channel)
Sample rate: Matches output_sample_rate specified in init

`output_text_boundary`

Timing information that maps a position in the output audio stream back to positions in the transcription and translation text. Use this to synchronize text highlighting with audio playback. Sent by default. Clients that don’t need word-level highlighting can opt out by setting the "hide_output_text_boundary" feature in configure, which suppresses these messages.

{
  "type": "output_text_boundary",
  "utterance_idx": 0,
  "output_time": 1.5,
  "transcription": {"word_idx": 3, "char_idx": 15},
  "translation": {"word_idx": 2, "char_idx": 12}
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance
`output_time`	float	Timestamp in seconds within the output audio stream
`transcription`	TextPosition or null	Position in the transcription text that aligns with the output audio at `output_time`
`translation`	TextPosition or null	Position in the translation text that aligns with the output audio at `output_time`. All translation before this position has already been spoken in the output audio.

TextPosition object:

Field	Type	Description
`word_idx`	integer	Index of the word within the utterance’s word list
`char_idx`	integer	Character offset within the word

`output_speech_ended`

Indicates all synthesized audio for an utterance has been sent.

{
  "type": "output_speech_ended",
  "utterance_idx": 0,
  "output_time": 4.2
}

Field	Type	Description
`utterance_idx`	integer	Index of the utterance
`output_time`	float	Final timestamp of the output audio

`flushed`

Confirms that a flush has been fully processed and all pending output has been sent.

{
  "type": "flushed",
  "request_id": "f1"
}

Field	Type	Description
`request_id`	string or null	Echoes the `request_id` from the `flush` message, if provided

`error`

Indicates an error occurred during processing.

{
  "type": "error",
  "message": "Invalid config message",
  "code": "CORE_ERROR",
  "utterance_idx": 3,
  "request_id": "r1"
}

Field	Type	Description
`message`	string	Human-readable error description
`code`	string	Error code identifier (e.g., `"STREAM_ERROR"` from the proxy, or an application-specific code from the backend)
`utterance_idx`	integer or null	Utterance associated with the error, if applicable
`request_id`	string or null	Associated request ID, if applicable

Non-fatal errors (e.g., malformed JSON, unknown message type) are sent as error messages with code "STREAM_ERROR" but the connection remains open. You can continue sending messages after receiving an error. Errors originating from the backend translation pipeline will have their own error codes.

Examples

Simultaneous Mode (default)

In simultaneous mode, the server produces translation and audio output in real-time as the speaker talks, with minimal delay. Setup:

{"type": "init", "realtime_playback": true}

{"type": "configure",
 "language_routes": [{"lang_in": "en-US", "lang_out": "es-ES"}]}

Flow:

Send init with "realtime_playback": true, then configure
Wait for configured
Stream input audio over the selected transport continuously
Receive streaming transcription, translation, and output audio in real time
The server uses VAD to segment utterances automatically; each utterance gets its own utterance_idx
Use flush when you want to force-finalize any pending output

Consecutive Mode with VAD (Automatic End-of-Speech)

In this mode, the speaker talks, the system detects when they stop, and then produces the full translation. This is the simplest approach for turn-based conversations. Setup:

{"type": "init"}

{"type": "configure",
 "language_routes": [{"lang_in": "en-US", "lang_out": "es-ES"}],
 "features": ["consecutive"]}

Flow:

Send init, then configure with "features": ["consecutive"]
Wait for configured
Stream input audio over the selected transport as the speaker talks
The server’s VAD detects end-of-speech and sends input_speech_ended
Receive transcription, translation, and output audio
Receive output_speech_ended when all audio for the utterance has been sent
For a multi-turn conversation, send a new configure to start the next turn (you can swap languages), or just keep streaming audio for the next utterance

Example message flow:

Client: {"type": "init", "input_sample_rate": 16000, "output_sample_rate": 24000}
Client: {"type": "configure", "language_routes": [{"lang_in": "en-US", "lang_out": "es-ES"}], "features": ["consecutive"], "request_id": "r1"}
Server: {"type": "configured", "request_id": "r1"}
Client: [input audio over transport]  (repeated)
Server: {"type": "input_speech_started", "utterance_idx": 0, "input_time": 0.3}
Server: {"type": "transcription", "utterance_idx": 0, "complete": [...], "partial": [...]}
Server: {"type": "input_speech_ended", "utterance_idx": 0, "input_time": 2.8}
Server: {"type": "language_route", "utterance_idx": 0, "lang_in": "en-US", "lang_out": "es-ES", "is_final": true}
Server: {"type": "transcription", "utterance_idx": 0, "complete": [...], "partial": []}
Server: {"type": "transcription_ended", "utterance_idx": 0}
Server: {"type": "translation", "utterance_idx": 0, "complete": [...], "partial": []}
Server: {"type": "translation_ended", "utterance_idx": 0}
Server: {"type": "output_speech_started", "utterance_idx": 0, "output_time": 0.0}
Server: [output audio over transport]  (repeated)
Server: {"type": "output_text_boundary", ...}
Server: {"type": "output_speech_ended", "utterance_idx": 0, "output_time": 3.5}

Consecutive Mode with Manual Flush

Use manual flush when you want explicit control over when the system processes the audio (e.g., push-to-talk with a button release, or after a fixed recording duration). Setup:

{"type": "init"}

{"type": "configure",
 "language_routes": [{"lang_in": "en-US", "lang_out": "es-ES"}],
 "features": ["consecutive"]}

Flow:

Send init, then configure with "features": ["consecutive"]
Wait for configured
Stream input audio over the selected transport while the user is speaking
When the user finishes (e.g., releases push-to-talk button), send flush
Receive transcription, translation, and audio output
Receive flushed when all output is complete
To start the next turn, simply resume sending audio (or send a new configure to change languages)

Example message flow:

Client: {"type": "init", "input_sample_rate": 16000, "output_sample_rate": 24000}
Client: {"type": "configure", "language_routes": [{"lang_in": "en-US", "lang_out": "es-ES"}], "features": ["consecutive"], "request_id": "r1"}
Server: {"type": "configured", "request_id": "r1"}
Client: [input audio over transport]  (repeated)
Client: {"type": "flush", "request_id": "f1"}
Server: {"type": "transcription", "utterance_idx": 0, "complete": [...], "partial": []}
Server: {"type": "translation", "utterance_idx": 0, "complete": [...], "partial": []}
Server: [output audio over transport]  (repeated)
Server: {"type": "flushed", "request_id": "f1"}

Multiple turns on the same connection: After receiving flushed, you can immediately stream more audio and send another flush for the next turn. You do not need to re-send init or configure (unless you want to change settings). This avoids reconnection overhead.

(... first turn completed with flushed ...)
Client: [input audio over transport]  (next turn)
Client: {"type": "flush", "request_id": "f2"}
Server: {"type": "transcription", ...}
Server: {"type": "translation", ...}
Server: [output audio over transport]
Server: {"type": "flushed", "request_id": "f2"}

Auto-Detect Mode (Any Language to One Target)

Use a wildcard "*" as lang_in to automatically detect the spoken language and translate it to a fixed target. This is ideal for scenarios like a help desk where callers may speak any language and all input should be translated to English: Setup:

{"type": "init"}

{"type": "configure",
 "language_routes": [{"lang_in": "*", "lang_out": "en-US"}],
 "features": ["consecutive"],
 "request_id": "r1"}

Flow:

Send init, then configure with a wildcard lang_in
Wait for configured
Stream input audio over the selected transport — the speaker can talk in any supported language
The server identifies the language and sends a language_route event with the detected language. You may receive intermediate is_final: false messages while detection is still refining, followed by the final is_final: true message once the route is locked:
{"type": "language_route", "utterance_idx": 0, "lang_in": "ko-KR", "lang_out": "en-US", "is_final": true}
Receive transcription (in the original language), translation (in English), and English output audio
Display the detected language to the user so they know it was recognized correctly

You can also combine auto-detect with explicit routes:

{"type": "configure",
 "language_routes": [
   {"lang_in": "en-US", "lang_out": "es-ES"},
   {"lang_in": "*", "lang_out": "en-US"}
 ],
 "features": ["consecutive"]}

In this example, English input is translated to Spanish. Any other language is translated to English.

Bidirectional Language Detection

For two-way conversations where either party may speak either language:

{"type": "configure",
 "language_routes": [
   {"lang_in": "en-US", "lang_out": "es-ES"},
   {"lang_in": "es-ES", "lang_out": "en-US"}
 ],
 "features": ["consecutive"]}

The server automatically detects which language is being spoken and routes to the appropriate translation direction. You will receive a language_route message for each utterance indicating which route was chosen.

Mid-Session Reconfiguration

You can change languages, voice, glossary, or features at any time by sending a new configure message. This is useful for:

Swapping speaker/language direction between turns
Changing the output voice
Updating the glossary

The server will tear down the current pipeline and build a new one. Wait for configured before sending new audio.

Client: {"type": "configure", "language_routes": [{"lang_in": "es-ES", "lang_out": "en-US"}], "features": ["consecutive"], "request_id": "r2"}
Server: {"type": "configured", "request_id": "r2"}
Client: [input audio over transport]

Supported Languages

Language codes use BCP-47 format (e.g., "en-US", "es-ES", "fr-FR"). To retrieve the current list of supported languages, use the Gateway languages API:

POST https://api.sanaslt.com/v2/languages/list

Error Handling

Connection Errors

Authentication is handled by the transport before the message layer starts. If authentication fails, the server returns an HTTP error before accepting the WebSocket connection:

HTTP Status	Description
401	Missing, empty, invalid, or expired authentication credentials

Runtime Errors

During a session, errors are delivered as error messages over the active Stream transport. Non-fatal errors (bad JSON, unknown message type) do not close the connection — you can continue sending messages. Fatal errors (backend connection failure, gRPC stream errors) may be followed by the connection closing.

Inactivity Timeout

If no messages or media are sent or received for 60 seconds (configurable), the server closes the transport session.

Best Practices

Audio chunking: Send audio in small chunks (10-20 ms of audio per chunk) for the lowest latency. Larger chunks add latency equal to the chunk duration.
Real-time pacing: When streaming from a live microphone, audio is naturally paced. When streaming from a file, pace sends to match real-time (e.g., sleep 20 ms between 20 ms chunks) for optimal results. Set realtime_playback: true in init when playing audio back in real-time.
Handle partial results: Display partial transcriptions and translations for a responsive UX, but only persist complete words. Complete words will not change and are sent only once.
Buffer output audio: Accumulate a small buffer of output audio before starting playback to prevent stuttering. Most client-side audio playback libraries handle this automatically.
Use request_id: Include request_id in configure and flush messages to correlate responses, especially when sending multiple operations.
Reconnection logic: Implement exponential backoff for connection failures.
Graceful shutdown: Send flush before closing the connection to ensure all pending output is received.
Reuse sessions: For multi-turn conversations, reuse the same transport session and send new configure messages between turns rather than reconnecting.

Migration Guide: v2 Consecutive API to v3 Stream API

This guide helps you migrate from the v2 /v2/consecutive WebSocket endpoint to the v3 /v3/stream WebSocket endpoint. The message shape changes also apply to clients using the WebRTC data channel for Stream v3 messages.

Endpoint Change

	v2	v3
URL	`wss://api.sanaslt.com/v2/consecutive`	`wss://api.sanaslt.com/v3/stream`
Auth	Same query params (`token`, `api_key`)	Same query params (`token`, `api_key`)
Auth failure	WebSocket accepted, then closed with code `1008`	HTTP `401` returned before WebSocket upgrade

Important: In v2, authentication failures happened after the WebSocket was established (close code 1008). In v3, authentication is validated before the upgrade, so failures result in a standard HTTP 401 response. Update your client error handling accordingly — you will receive an HTTP error, not a WebSocket close frame.

Protocol Changes

The biggest change is that v3 separates session initialization into two explicit steps (init + configure) instead of a single config message.

Initialization: `config` -> `init` + `configure`

v2: Single config message

{
  "type": "config",
  "lang_in": "en-US",
  "lang_out": "es-ES",
  "input_sample_rate": 16000,
  "output_sample_rate": 24000,
  "glossary": ["Sanas", "ACME"],
  "can_lang_swap": false
}

v3: Two messages (init + configure)

{
  "type": "init",
  "input_sample_rate": 16000,
  "output_sample_rate": 24000
}

{
  "type": "configure",
  "language_routes": [
    {"lang_in": "en-US", "lang_out": "es-ES"}
  ],
  "features": ["consecutive"],
  "glossary": [
    {"terms": {"*": "Sanas"}},
    {"terms": {"*": "ACME"}}
  ],
  "request_id": "r1"
}

Key differences:

Sample rates moved to init (sent once per connection)
lang_in/lang_out replaced by the language_routes array
can_lang_swap: true replaced by adding a second route with swapped languages
glossary changed from a flat string array to an array of {"terms": {...}} objects. Use "*" as the language key to apply a term across all languages (recommended default), and override specific languages as needed.
You must include "features": ["consecutive"] to get consecutive mode behavior
voice_id remains available on configure (same as v2 config)

Ready Signal: `ready` -> `configured`

v2:

{"type": "ready", "session_id": "abc123"}

v3:

{"type": "configured", "request_id": "r1"}

The session_id field is no longer provided in the ready response. If you need a session identifier, use the conversation_id from init.

Audio Messages: JSON `audio` -> Binary WebSocket frames

v2 (input):

{"type": "audio", "data": "<base64>"}

v3 (input): Raw PCM bytes sent as a WebSocket Binary frame (no JSON wrapper, no base64 encoding). v2 (output):

{"type": "audio", "data": "<base64>"}

v3 (output): Raw PCM bytes received as a WebSocket Binary frame (no JSON wrapper, no base64 encoding). In v2, both input and output audio used JSON text frames with base64-encoded data. In v3, audio is sent and received as raw binary WebSocket frames, eliminating the ~33% base64 overhead. All other messages remain JSON text frames.

Transcription and Translation: added `utterance_idx`

v2 transcription and translation messages contained only complete and partial word arrays. v3 adds an utterance_idx field to each message so you can associate results with a specific utterance when multiple utterances are in flight. v2:

{"type": "transcription", "complete": [...], "partial": [...]}

v3:

{"type": "transcription", "utterance_idx": 0, "complete": [...], "partial": [...]}

End-of-Speech: `stop`/`flush` / `speech_stop` -> `flush` / `flushed` + lifecycle events

v2 (client sends):

{"type": "stop"}

or:

{"type": "flush"}

Both stop and flush were accepted in v2 and had identical behavior. In v3, only flush exists, and it now accepts an optional request_id for correlation. v3 (client sends):

{"type": "flush", "request_id": "f1"}

v2 (server confirms end-of-speech via VAD):

{"type": "speech_stop", "utterance_idx": 0}

v3 (server lifecycle events):

{"type": "input_speech_ended", "utterance_idx": 0, "input_time": 2.8}

{"type": "output_speech_ended", "utterance_idx": 0, "output_time": 3.5}

In v3, the server provides more granular lifecycle events. The speech_stop message is replaced by input_speech_ended (when the server detects the speaker stopped) and output_speech_ended (when all output audio has been sent). The flushed message confirms a manual flush is complete. Note: in v2, a flushed message could arrive without a client-initiated stop/flush (triggered automatically by VAD via output_speech_ended). In v3, output_speech_ended and flushed are distinct events — flushed only arrives in response to a client flush.

Language Detection: `languages` -> `language_route`

v2:

{"type": "languages", "lang_in": "en-US", "lang_out": "es-ES"}

v3:

{"type": "language_route", "utterance_idx": 0, "lang_in": "en-US", "lang_out": "es-ES", "is_final": true}

The v3 version includes utterance_idx to associate the language detection with a specific utterance, and is_final to distinguish the locked-in route from tentative routes emitted while language identification is still refining.

`can_lang_swap` -> Multiple Language Routes

v2:

{"type": "config", "lang_in": "en-US", "lang_out": "es-ES", "can_lang_swap": true}

v3:

{"type": "configure", "language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"},
  {"lang_in": "es-ES", "lang_out": "en-US"}
]}

Instead of a boolean flag, v3 uses explicit bidirectional routes. You can also add a wildcard "*" -> "*" catch-all route to detect unexpected languages and prompt the user to switch (see Language Routes for details):

{"type": "configure", "language_routes": [
  {"lang_in": "en-US", "lang_out": "es-ES"},
  {"lang_in": "es-ES", "lang_out": "en-US"},
  {"lang_in": "*", "lang_out": "*"}
]}

Glossary Format

v2:

{"glossary": ["Sanas", "ACME"]}

Each string was treated as a term that should be preserved identically across all languages. v3:

{"glossary": [
  {"terms": {"*": "Sanas"}},
  {"terms": {"*": "ACME"}},
  {"terms": {"*": "customer service", "es-ES": "servicio al cliente"}}
]}

The v3 format uses structured objects with per-language mappings. Always include a "*" wildcard key as the default so the term is handled correctly for any language. You can override specific languages alongside the wildcard. For brand names and proper nouns that should never change, a single "*" entry is all you need.

Speech Delimiter: `speech_delimiter` -> `output_text_boundary`

v2:

{
  "type": "speech_delimiter",
  "time": 1.5,
  "transcription": {"utterance_idx": 0, "word_idx": 3, "char_idx": 15},
  "translation": {"utterance_idx": 0, "word_idx": 2, "char_idx": 12}
}

v3:

{
  "type": "output_text_boundary",
  "utterance_idx": 0,
  "output_time": 1.5,
  "transcription": {"word_idx": 3, "char_idx": 15},
  "translation": {"word_idx": 2, "char_idx": 12}
}

The utterance_idx moved to the top level, time was renamed to output_time, and the inner position objects no longer repeat utterance_idx. v3 emits boundaries by default (matching v2’s unconditional speech_delimiter), and clients that don’t need them can opt out by adding "hide_output_text_boundary" to the configure.features list to skip server-side alignment work.

Renamed Events

v2 Event	v3 Event	Notes
`translation_end`	`translation_ended`	Name changed for consistency. Now includes `utterance_idx`.

New Events in v3

The following server events are new in v3 and were not exposed on the v2 WebSocket:

Event	Description
`input_speech_started`	VAD detected the start of speech
`transcription_ended`	All transcription for an utterance is complete
`output_speech_started`	Server began sending output audio for an utterance
`output_speech_ended`	All output audio for an utterance has been sent

Quick Migration Checklist

Change WebSocket URL from /v2/consecutive to /v3/stream
Replace the single config message with init + configure
Add "features": ["consecutive"] to the configure message
Replace JSON "type": "audio" input messages with raw PCM bytes in WebSocket Binary frames
Handle output audio as WebSocket Binary frames containing raw PCM bytes (instead of JSON "type": "audio" text frames)
Replace "type": "stop" (or "type": "flush") with "type": "flush" (now supports request_id)
Handle "type": "configured" instead of "type": "ready"
Handle "type": "language_route" instead of "type": "languages"
Handle "type": "input_speech_ended" instead of "type": "speech_stop"
Handle "type": "output_text_boundary" instead of "type": "speech_delimiter" (emitted by default like v2; add "hide_output_text_boundary" to configure.features to opt out)
Update glossary format from flat strings to {"terms": {"*": "term"}} objects (always include the "*" wildcard key)
Replace can_lang_swap: true with a second language route
Handle utterance_idx on transcription and translation messages (new in v3)
Handle "type": "translation_ended" (renamed from v2 "type": "translation_end")
(Optional) Handle new lifecycle events for richer UX

​Protocol Overview

​Connection Lifecycle

​Key Concepts

​Client Messages (Client -> Server)

​init

​configure

​Language Routes

​Glossary Entries

​Features

​Input Audio

​flush

​Server Messages (Server -> Client)

​configured

​input_speech_started

​language_route

​transcription

​transcription_ended

​translation

​translation_ended

​input_speech_ended

​output_speech_started

​Output Audio

​output_text_boundary

​output_speech_ended

​flushed

​error

​Examples

​Simultaneous Mode (default)

​Consecutive Mode with VAD (Automatic End-of-Speech)

​Consecutive Mode with Manual Flush

​Auto-Detect Mode (Any Language to One Target)

​Bidirectional Language Detection

​Mid-Session Reconfiguration

​Supported Languages

​Error Handling

​Connection Errors

​Runtime Errors

​Inactivity Timeout

​Best Practices

​Migration Guide: v2 Consecutive API to v3 Stream API

​Endpoint Change

​Protocol Changes

​Initialization: config -> init + configure

​Ready Signal: ready -> configured

​Audio Messages: JSON audio -> Binary WebSocket frames

​Transcription and Translation: added utterance_idx

​End-of-Speech: stop/flush / speech_stop -> flush / flushed + lifecycle events

​Language Detection: languages -> language_route

​can_lang_swap -> Multiple Language Routes

​Glossary Format

​Speech Delimiter: speech_delimiter -> output_text_boundary

​Renamed Events

​New Events in v3

​Quick Migration Checklist

Protocol Overview

Connection Lifecycle

Key Concepts

Client Messages (Client -> Server)

`init`

`configure`

Language Routes

Glossary Entries

Features

Input Audio

`flush`

Server Messages (Server -> Client)

`configured`

`input_speech_started`

`language_route`

`transcription`

`transcription_ended`

`translation`

`translation_ended`

`input_speech_ended`

`output_speech_started`

Output Audio

`output_text_boundary`

`output_speech_ended`

`flushed`

`error`

Examples

Simultaneous Mode (default)

Consecutive Mode with VAD (Automatic End-of-Speech)

Consecutive Mode with Manual Flush

Auto-Detect Mode (Any Language to One Target)

Bidirectional Language Detection

Mid-Session Reconfiguration

Supported Languages

Error Handling

Connection Errors

Runtime Errors

Inactivity Timeout

Best Practices

Migration Guide: v2 Consecutive API to v3 Stream API

Endpoint Change

Protocol Changes

Initialization: `config` -> `init` + `configure`

Ready Signal: `ready` -> `configured`

Audio Messages: JSON `audio` -> Binary WebSocket frames

Transcription and Translation: added `utterance_idx`

End-of-Speech: `stop`/`flush` / `speech_stop` -> `flush` / `flushed` + lifecycle events

Language Detection: `languages` -> `language_route`

`can_lang_swap` -> Multiple Language Routes

Glossary Format

Speech Delimiter: `speech_delimiter` -> `output_text_boundary`

Renamed Events

New Events in v3

Quick Migration Checklist