> ## Documentation Index
> Fetch the complete documentation index at: https://developer.sanas.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Browser SDK - Developer Guide

The Sanas Browser SDK lets a web application send a live microphone stream through Sanas's audio processing (accent translation, speech enhancement, language translation) and receive the processed audio back as a `MediaStream`. It runs entirely in the browser and is transport-agnostic from your point of view: give it an input stream, receive an output stream.

This guide covers everything a developer needs to ship an integration. If you get stuck, jump to [Troubleshooting](#troubleshooting).

***

## Contents

1. [Overview](#overview)
2. [Installation](#installation)
3. [Authentication](#authentication)
4. [Quick start](#quick-start)
5. [Remote Inferencing mode (accent translation, speech enhancement)](#remote-inferencing-mode)
6. [Language Translation mode](#language-translation-mode)
7. [The processor lifecycle](#the-processor-lifecycle)
8. [API reference](#api-reference)
9. [Error handling](#error-handling)
10. [Advanced options](#advanced-options)
11. [Troubleshooting](#troubleshooting)

***

## Overview

The SDK exposes a single top-level object — `SanasSDK` — that manages a session. From that session you create one or more `AudioProcessor` instances, each of which:

* Takes a `MediaStream` in (typically from `navigator.mediaDevices.getUserMedia`).
* Emits a processed `MediaStream` out. You can attach the output to an `<audio>` element, feed it into WebRTC, or hand it to any other Web Audio API consumer.
* Runs one of two modes:
  * **Remote Inferencing (RI)** — a single Sanas model applied to the stream. Used for accent translation, speech enhancement, and other single-model features. You identify the model by name.
  * **Language Translation (LT)** — one-direction live translation. Each processor handles a single direction (say English → Spanish). A real two-person conversation uses **two processors** — one per speaker's direction — linked by a shared `conversationId` so the server can share context between them. Callbacks stream per-utterance transcripts and language-detection hints back.

Availability of RI and LT is controlled per group in the Sanas Developer Console. The SDK checks entitlement before creating a processor and throws a distinct error if a feature isn't enabled for your group.

## Installation

```text theme={null}
npm install @sanas-ai/browser-sdk
```

The package is self-contained — no peer dependencies. Works with any modern bundler (Vite, webpack, Next.js, Rollup, Parcel).

**Browser support:** Any browser that supports `MediaStream`, `RTCPeerConnection`, and `WebSocket` — Chrome 90+, Firefox 90+, Safari 15+, Edge 90+.

## Authentication

The SDK never sees your Sanas API key. Instead, it accepts a `tokenProvider` callback that asks *your* backend for a short-lived access token, and your backend uses its stored API key to mint one.

**Never embed a Sanas API key in browser code.** Anyone can view your bundle, extract the key, and use it against your usage budget.

### Backend responsibilities

Your backend needs a single endpoint (name it whatever you want) that:

1. Authenticates the browser request using your app's own session mechanism (cookies, JWT, etc. — that's your call).
2. Calls Sanas's OAuth token endpoint with your stored API key.
3. Returns the token payload to the browser as JSON.

Here's a minimal Node.js example (no dependencies beyond Node 18+):

```js theme={null}
import http from "node:http";

const SANAS_API_KEY = process.env.SANAS_API_KEY;
const SANAS_IDENTITY_URL = "https://identity.sanas.ai";
const ALLOWED_ORIGIN = process.env.ALLOWED_ORIGIN || "http://localhost:5173";

http.createServer(async (req, res) => {
  // CORS preflight
  if (req.method === "OPTIONS") {
    res.writeHead(204, corsHeaders());
    return res.end();
  }

  if (req.method === "POST" && req.url === "/token") {
    // Authenticate the calling browser session here (skipped for brevity).

    const upstream = await fetch(`${SANAS_IDENTITY_URL}/oauth2/api-key/login`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ apiKey: SANAS_API_KEY }),
    });

    const body = await upstream.text();
    res.writeHead(upstream.status, {
      "Content-Type": "application/json",
      ...corsHeaders(),
    });
    return res.end(body);
  }

  res.writeHead(404, corsHeaders());
  res.end();
}).listen(process.env.PORT || 4000);

function corsHeaders() {
  return {
    "Access-Control-Allow-Origin": ALLOWED_ORIGIN,
    "Access-Control-Allow-Methods": "POST, OPTIONS",
    "Access-Control-Allow-Headers": "Content-Type",
  };
}
```

Run it with `SANAS_API_KEY=... node server.js`.

### Browser side

```js theme={null}
import { SanasSDK, type TokenResponse } from "@sanas-ai/browser-sdk";

const sdk = new SanasSDK();

await sdk.initialize({
  tokenProvider: async (): Promise<TokenResponse> => {
    const res = await fetch("https://your-backend.example.com/token", {
      method: "POST",
      credentials: "include", // if you use cookie sessions
    });
    if (!res.ok) throw new Error(`Token request failed: ${res.status}`);
    return res.json();
  },
});
```

`tokenProvider` is called once at `initialize()` and again any time the SDK needs a fresh token. Make it idempotent and quick — the SDK will time it out after 30 seconds.

### Origin allowlisting

Every group in the Sanas Developer Console has an **allowed origins** list. All API calls the SDK makes from the browser include the page's origin; requests from a non-allowlisted origin are rejected. Add every origin your app runs on (production, staging, localhost during development) under **Project settings → Allowed origins**.

## Quick start

```js theme={null}
import { SanasSDK, ProcessorState } from "@sanas-ai/browser-sdk";

const sdk = new SanasSDK();

// 1. Initialize the SDK with your token provider.
await sdk.initialize({
  tokenProvider: () =>
    fetch("/token", { method: "POST" }).then((r) => r.json()),
});

// 2. Capture the microphone.
const input = await navigator.mediaDevices.getUserMedia({ audio: true });

// 3. Create a processor. RI mode with an accent-translation model:
const processor = await sdk.createAudioProcessor(
  { input, modelName: "AT5.2" },
  {
    // Drive your UI off state, not off the reason string.
    // `Initializing` — session is being set up; output is silent.
    // `Ready`        — audio is flowing through the processor.
    // `Failed`       — fatal error; the processor can't recover. Destroy it,
    //                  optionally show `reason` to the user, and create a new
    //                  one if you want to keep going.
    // `Disconnected` — you called destroyAudioProcessor(); terminal.
    onStateChange: (state, reason) => {
      if (state === ProcessorState.Ready) enableMicButton();
      if (state === ProcessorState.Failed) {
        disableMicButton();
        showBanner(reason);
      }
    },
  }
);

// 4. Attach the processed output to an <audio> element.
const audio = document.querySelector("audio")!;
audio.srcObject = processor.outputStream;
audio.play();

// 5. When you're done, tear down.
await sdk.destroyAudioProcessor(processor);
await sdk.shutdown();
```

## Remote Inferencing mode

For accent translation, speech enhancement, and similar single-model features. The model you use is provisioned in your Developer Console; the SDK verifies your group is entitled to the model before starting the call.

```js theme={null}
const processor = await sdk.createAudioProcessor(
  {
    input: micStream,
    modelName: "AT5.2", // as shown in the Developer Console
  },
  {
    onStateChange: (state, reason) => {
      if (state === ProcessorState.Ready) console.log("live");
      if (state === ProcessorState.Failed) console.error("failed:", reason);
    },
  }
);

audioElement.srcObject = processor.outputStream;
```

**Common errors:**

* `CreateProcessorErrorCode.FeatureNotEntitled` — Your group doesn't have Remote Inferencing enabled at all. Contact your Sanas administrator.
* `CreateProcessorErrorCode.ModelNotEntitled` — Your group has RI but not this specific model. Verify the exact name against the console.

## Language Translation mode

Each LT processor handles **one direction** of translation — one input language, one output language. A processor delivers translated audio out and streams two kinds of side-channel data via callbacks:

* `onTranscript` — a per-utterance transcript with partial and complete words for the detected input language.
* `onLanguageRoute` — a per-utterance language-detection result telling you which input language the server detected on that utterance (and which output language it will render).

### Two-person conversations use two processors

For a real conversation between two people who speak different languages, run **two processors on the same page**, one per direction, linked by a shared `conversationId`. For example, an English-speaking agent and a Spanish-speaking customer on the same call:

* Processor A: agent's mic (`languageIn: "en-US"`) → Spanish audio played to the customer (`languageOut: "es-ES"`).
* Processor B: customer's mic (`languageIn: "es-ES"`) → English audio played to the agent (`languageOut: "en-US"`).

Give both processors the same `conversationId` (any UUID you generate per call) and the server shares translation context between them — proper nouns, jargon, and prior context stay consistent across both directions.

```js theme={null}
const conversationId = crypto.randomUUID();

// Agent's mic → Spanish audio for the customer.
const outbound = await sdk.createAudioProcessor(
  {
    input: agentMicStream,
    languageIn: "en-US",
    languageOut: "es-ES",
    conversationId,
  },
  {
    onStateChange: (state, reason) => console.log("outbound", state, reason),
    onTranscript: (json) => console.log("outbound transcript", JSON.parse(json)),
    onLanguageRoute: (json) => console.log("outbound route", JSON.parse(json)),
  }
);
customerSpeakerElement.srcObject = outbound.outputStream;

// Customer's mic → English audio for the agent.
const inbound = await sdk.createAudioProcessor(
  {
    input: customerMicStream,
    languageIn: "es-ES",
    languageOut: "en-US",
    conversationId, // same UUID — shared context.
  },
  {
    onStateChange: (state, reason) => console.log("inbound", state, reason),
    onTranscript: (json) => console.log("inbound transcript", JSON.parse(json)),
    onLanguageRoute: (json) => console.log("inbound route", JSON.parse(json)),
  }
);
agentSpeakerElement.srcObject = inbound.outputStream;
```

`conversationId` is optional. Omit it if you're only running a single direction (e.g. a monologue translation) and don't need context sharing.

### Single-direction usage

If you only need to translate one direction (for example, translating a broadcast into another language), a single processor is enough:

```js theme={null}
const processor = await sdk.createAudioProcessor(
  {
    input: micStream,
    languageIn: "en-US",
    languageOut: "es-MX",
  },
  {
    onStateChange: (state, reason) => console.log(state, reason),
    onTranscript: (json) => console.log("transcript", JSON.parse(json)),
    onLanguageRoute: (json) => console.log("route", JSON.parse(json)),
  }
);
```

### `onTranscript` payload

Delivered as a JSON string; parse before use. **Fires multiple times per utterance** as the server refines its hypothesis. Each event carries a mix of `complete` (newly-finalized words) and `partial` (still-in-flight words).

```json theme={null}
{
  "type": "transcription",
  "transcription": {
    "lang": "en-US",
    "utterance_idx": 0,
    "complete": [],
    "partial": [
      { "word": "Hi",      "start": 0.0, "end": 0.0, "probability": 0.900 },
      { "word": ",",       "start": 0.0, "end": 0.0, "probability": 0.881 },
      { "word": " testing","start": 0.0, "end": 0.0, "probability": 0.941 },
      { "word": " of",     "start": 0.0, "end": 0.0, "probability": 0.936 }
    ]
  }
}
```

**How the stream evolves for one utterance.** `partial` is the current best guess and is replaced wholesale each event as the server refines it. `complete` is a **delta of newly-finalized words in this event only** — it is *not* the running total of everything finalized so far. To get the full utterance you must accumulate `complete` per `utterance_idx` on your side. `end` / `start` timestamps are reserved for future use and can be treated as 0.

For example, an utterance like "Hi, testing of language translation." arrives across several events. The `complete` deltas are `[]`, `[]`, `["Hi", ",", " testing", " of"]`, `[" language"]`, `[" translation"]`, `["."]` — concatenating those deltas rebuilds the full sentence.

Minimal accumulator:

```js theme={null}
const utterances = new Map<number, string>();

onTranscript: (json) => {
  const { transcription } = JSON.parse(json);
  const prev = utterances.get(transcription.utterance_idx) ?? "";
  const finalized = transcription.complete.map((w) => w.word).join("");
  const pending = transcription.partial.map((w) => w.word).join("");
  utterances.set(transcription.utterance_idx, prev + finalized);
  render(prev + finalized + pending); // finalized + current guess
};
```

Trailing punctuation typically arrives in its own event a second after the last word — plan your UI to accept it gracefully rather than assuming the utterance is done as soon as `partial` empties.

`utterance_idx` starts at 0 and increments per utterance. It correlates with `onLanguageRoute` events so you can label transcripts with the language the server detected.

### `onLanguageRoute` payload

Fires **once at the start of each utterance**, before the transcript events for that utterance begin arriving. Tells you the input language the server detected on that utterance and the output language it's rendering into.

```json theme={null}
{
  "type": "language_route_chosen",
  "language_route_chosen": {
    "utterance_idx": 1,
    "lang_in": "en-US",
    "lang_out": "es-ES",
    "is_final": true
  }
}
```

* `utterance_idx` — correlates with `onTranscript` events for the same utterance.
* `lang_in` / `lang_out` — the concrete input and output languages for this utterance.
* `is_final` — `true` when the server has locked in the routing for this utterance. In wildcard-mode edge cases the server may emit tentative routes with `is_final: false` before it commits; a subsequent event with the same `utterance_idx` and `is_final: true` supersedes the earlier one.

Fires for every utterance in both normal and wildcard mode. In **wildcard mode**, set `languageIn` or `languageOut` to `"*"` when creating the processor and the server detects the actual language for you; the value you get here is the concrete language the server chose. In **normal mode** (fixed `languageIn` / `languageOut`), the values echo back what the server confirmed for that utterance.

### Changing languages mid-call

Rather than tearing down and recreating the processor (which drops the call), swap languages in place:

```text theme={null}
await processor.updateLanguageConfig({
  languageIn: "en-US",
  languageOut: "fr-FR",
});
```

Usage attribution correctly rotates across language changes — the SDK bills each language segment separately without losing history when you switch back.

**Common error:**

* `CreateProcessorErrorCode.FeatureNotEntitled` — Language Translation isn't enabled for your group. Contact your Sanas administrator.

## The processor lifecycle

Every processor moves through a small state machine, observable via `onStateChange`:

```text theme={null}
Initializing ──► Ready ──► Disconnected (consumer destroyed)
    │              │
    │              └──► Failed (mid-call error)
    │
    └──► Failed (setup error)
```

* **Initializing** — the SDK is negotiating the media session. `outputStream` exists and is silent.
* **Ready** — the media session is up. `outputStream` is now delivering processed audio.
* **Failed** — a fatal error occurred. Read `reason` for a short human-readable explanation. The processor cannot be recovered; destroy it and create a new one.
* **Disconnected** — the consumer called `destroyAudioProcessor`. Terminal, expected.

The `outputStream` reference is **stable for the life of the processor** — you can attach it to an `<audio>` element or WebRTC track before the processor reaches `Ready`, and it will start emitting audio automatically once the media path opens.

## API reference

### `SanasSDK`

```ruby theme={null}
class SanasSDK {
  initialize(params: InitParams): Promise<void>;
  isInitialized(): boolean;
  createAudioProcessor(
    params: AudioProcessorParams,
    callbacks?: ProcessorCallbacks
  ): Promise<IAudioProcessor>;
  destroyAudioProcessor(processor: IAudioProcessor): Promise<void>;
  getActiveProcessorCount(): number;
  shutdown(): Promise<void>;
}
```

#### `initialize(params)`

Establish a session with your token provider and validate your group's entitlements. Must be called before `createAudioProcessor`. Calling `initialize` twice throws `InitErrorCode.AlreadyInitialized`.

#### `createAudioProcessor(params, callbacks?)`

Create a processor for a single audio stream. You can hold multiple active processors on one SDK instance (each processes an independent stream).

Wire callbacks at construction — they are guaranteed to fire even for the initial `Initializing → Failed` transition on setup errors.

#### `destroyAudioProcessor(processor)`

Gracefully tear down a single processor. The output stream ends. Emits a final usage snapshot before returning so that usage accounting is complete.

#### `shutdown()`

Destroy all live processors, stop background reporting, clear all session state. Call this on page unload or when your app no longer needs Sanas. After `shutdown`, you can call `initialize` again to start a new session.

### `InitParams`

```typescript theme={null}
interface InitParams {
  tokenProvider: () => Promise<TokenResponse>;
  logLevel?: "debug" | "info" | "warn" | "error" | "silent";
  jitterBufferTargetMs?: number;
}
```

### `TokenResponse`

```typescript theme={null}
interface TokenResponse {
  access_token: string;
  user_id: string;
  group_id: string;
  expires_in?: number;
  authenticated_at?: string;
  token_type?: string;
}
```

This is exactly the shape returned by Sanas's OAuth login endpoint — your backend can forward the response body verbatim.

### `AudioProcessorParams`

A discriminated union:

```typescript theme={null}
type AudioProcessorParams =
  | AudioProcessorModelParams
  | AudioProcessorLanguageParams;

interface AudioProcessorModelParams {
  input: MediaStream;
  modelName: string;
  sampleRate?: number;
  jitterBufferTargetMs?: number;
}

interface AudioProcessorLanguageParams {
  input: MediaStream;
  languageIn: string;
  languageOut: string;
  conversationId?: string;
  sampleRate?: number;
  jitterBufferTargetMs?: number;
}
```

### `ProcessorCallbacks`

```typescript theme={null}
interface ProcessorCallbacks {
  onStateChange?: (state: ProcessorState, reason: string) => void;
  onTranscript?: (json: string) => void;
  onLanguageRoute?: (json: string) => void;
}
```

The two JSON callbacks fire only for LT processors. The payload is a raw JSON string — parse with `JSON.parse` on your side.

### `IAudioProcessor`

```typescript theme={null}
interface IAudioProcessor {
  readonly outputStream: MediaStream;
  getState(): ProcessorState;
  updateInputStream(input: MediaStream): Promise<void>;
  updateLanguageConfig(config: {
    languageIn: string;
    languageOut: string;
    conversationId?: string;
  }): Promise<void>;
}
```

`updateInputStream` swaps the mic mid-call (device change, headset unplug) without dropping the session.

`updateLanguageConfig` throws if called on a model-mode processor.

### `ProcessorState`

```text theme={null}
enum ProcessorState {
  Initializing = "initializing",
  Ready = "ready",
  Failed = "failed",
  Disconnected = "disconnected",
  Unknown = "unknown",
}
```

## Error handling

All SDK errors are instances of `SdkError` with a machine-readable `code` and a human-readable `message`.

```js theme={null}
import { SdkError, InitErrorCode } from "@sanas-ai/browser-sdk";

try {
  await sdk.initialize({ tokenProvider });
} catch (err) {
  if (err instanceof SdkError && err.code === InitErrorCode.OriginNotAllowed) {
    showBanner("This origin isn't allowed. Contact your admin.");
  } else {
    throw err;
  }
}
```

### `InitErrorCode`

| Code                       | Meaning                                                                   | Recommended action                                                                                                       |
| -------------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| `AlreadyInitialized`       | `initialize` was called on an already-initialized SDK.                    | Call `shutdown()` first, or reuse the existing session.                                                                  |
| `InitializationInProgress` | A concurrent `initialize` is still running.                               | Await the first call before starting another.                                                                            |
| `InvalidParameters`        | `tokenProvider` was missing or not a function.                            | Check the call site.                                                                                                     |
| `AuthenticationFailed`     | Your backend returned an error, or the token payload was malformed.       | Log the message; verify your backend is returning `{access_token, user_id, group_id, ...}`.                              |
| `OriginNotAllowed`         | This page's origin isn't in the group's allowed list.                     | Add the origin in **Project settings → Allowed origins**.                                                                |
| `NetworkError`             | Network / CORS failure reaching either your backend or the Sanas console. | Message includes which stage failed; check CORS on your backend or inspect DevTools network tab for the console request. |
| `UnknownError`             | Anything else.                                                            | Read the message; open a support ticket if it recurs.                                                                    |

### `CreateProcessorErrorCode`

| Code                    | Meaning                                                       | Recommended action                                       |
| ----------------------- | ------------------------------------------------------------- | -------------------------------------------------------- |
| `SdkNotInitialized`     | You didn't call `initialize`, or the session was lost.        | Initialize (or reinitialize) before creating processors. |
| `InvalidParameters`     | Required field missing or unsafe characters in a param.       | Check `modelName` / `languageIn` / `languageOut` values. |
| `UnsupportedSampleRate` | The requested sample rate isn't supported.                    | Use 8000, 16000, or 48000.                               |
| `ModelNotEntitled`      | The model name isn't in this group's entitlements.            | Verify the exact name in the console. Case-sensitive.    |
| `FeatureNotEntitled`    | The feature (RI or LT) isn't enabled at group level.          | Contact your Sanas administrator.                        |
| `OriginNotAllowed`      | Same as init — origin got revoked between init and this call. | Update **Project settings → Allowed origins**.           |
| `NetworkError`          | Console API unreachable for the entitlement check.            | Retry; check DevTools.                                   |
| `UnknownError`          | Anything else.                                                | Read the message.                                        |

### Runtime state changes

Once a processor is created, transient failures surface through `onStateChange(Failed, reason)`, not by throwing. Wire the callback at construction so you never miss an early failure:

```js theme={null}
sdk.createAudioProcessor(params, {
  onStateChange: (state, reason) => {
    if (state === ProcessorState.Failed) {
      // reason is a short user-facing string
      // e.g. "Server rejected the call (auth failure)"
      // e.g. "Media connection lost"
      // e.g. "Authentication lost: origin not allowed for this group"
      // e.g. "Entitlement revoked: language_translation is no longer available for this group"
      showErrorBanner(reason);
    }
  },
});
```

## Advanced options

### Sample rate

Default is 16000 Hz. Supported: 8000 / 16000 / 48000. Set per processor:

```text theme={null}
sdk.createAudioProcessor({ input, modelName: "...", sampleRate: 48000 }, ...);
```

Rates outside the supported set throw `UnsupportedSampleRate`.

### Jitter buffer

For bumpy networks, increase the receive-side buffer to smooth playback at the cost of extra latency. Default is chosen by the SDK; override globally in `InitParams` or per processor:

```text theme={null}
await sdk.initialize({ tokenProvider, jitterBufferTargetMs: 200 });
// or
sdk.createAudioProcessor(
  { input, modelName: "...", jitterBufferTargetMs: 100 },
  ...
);
```

### Swapping input mid-call

Users unplug headsets or switch devices. Update in place instead of tearing down:

```js theme={null}
const newInput = await navigator.mediaDevices.getUserMedia({
  audio: { deviceId: newDeviceId },
});
await processor.updateInputStream(newInput);
```

The active call stays up; the outgoing track is swapped.

### Multiple concurrent processors

You can run more than one processor per SDK instance — e.g. a mic in RI mode and a system-audio stream in LT mode. Each is fully independent.

### Logging

Set `logLevel: "debug"` in `InitParams` to see the SDK's internal logs in DevTools. Default is `warn`. Use `silent` in production if you want no console output.

## Troubleshooting

\*\*"Origin not allowed" during \*\*`initialize`<br />Your page's origin (protocol + host + port) isn't in the group's allowed list. Add it under **Project settings → Allowed origins**. Note that `http://localhost:5173` and `http://localhost:3000` are different origins — add every dev port you use.

**Processor reaches `Failed` immediately with "Server rejected the call (auth failure)"**<br />The access token expired between `tokenProvider` returning it and the media session opening, or your backend returned a stale token.

**Processor reaches `Failed` mid-call with "Media connection lost"**<br />Network dropped. The SDK does not auto-reconnect a lost media session. Destroy the processor and create a new one.

**Metrics appear delayed**<br />The SDK batches usage reports on a 1-minute cadence. If you tear down a processor, the final snapshot is captured before the underlying session closes.