Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## [4.43.4]

- Add `assemblyai-stt-3-7-realtime` (note taking) and `assemblyai-stt-3-7-realtime-agent` (voice agents) to the streaming `speechModel` options (`StreamingSpeechModel`)
- Add `"manual"` to the streaming `StreamingMode` type: the client ends each turn with `forceEndpoint()`. Only `assemblyai-stt-3-7-realtime-agent` supports it

## [4.43.3]

- `StreamingTranscriber` emits a `termination` event with the server's `Termination` message (`TerminationEvent`: `audio_duration_seconds`, `session_duration_seconds`), and `close()` resolves with it, or with `undefined` when it did not wait for the message or the message did not arrive before the timeout or the socket closed. `close()` now calls the `close` listener for a client-initiated close (code `1000`, reason `"Session ended"` or `"Closed by the client"`) before it clears the listeners; it is called once per session. Code that types `close()`'s result as `Promise<void>` must widen it to `Promise<TerminationEvent | undefined>`
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "assemblyai",
"version": "4.43.3",
"version": "4.43.4",
"description": "The AssemblyAI JavaScript SDK provides an easy-to-use interface for interacting with the AssemblyAI API, which supports async and real-time transcription and text-to-speech.",
"engines": {
"node": ">=18"
Expand Down
8 changes: 7 additions & 1 deletion src/types/streaming/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,11 +234,17 @@ export type StreamingSpeechModel =
| "universal-3-5-pro"
| "universal-3-6"
| "universal-3-6-pro"
| "assemblyai-stt-3-7-realtime"
| "assemblyai-stt-3-7-realtime-agent"
| "u3-pro";

export type StreamingDomain = "medical-v1" | "veterinary-v1";

export type StreamingMode = "max_accuracy" | "min_latency" | "balanced";
export type StreamingMode =
| "max_accuracy"
| "min_latency"
| "balanced"
| "manual";

export type VoiceFocusModel = "near-field" | "far-field";

Expand Down
52 changes: 52 additions & 0 deletions tests/unit/streaming.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -399,6 +399,24 @@ describe("streaming", () => {
await connect(rt, server);
});

it("should include mode=manual in connection URL", async () => {
await cleanup();
WS.clean();

const wsUrl = `${websocketBaseUrl}?token=123&sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent&mode=manual`;
server = new WS(wsUrl);
rt = new StreamingTranscriber({
websocketBaseUrl,
token: "123",
sampleRate: 16_000,
speechModel: "assemblyai-stt-3-7-realtime-agent",
mode: "manual",
});
onOpen = jest.fn();
rt.on("open", onOpen);
await connect(rt, server);
});

it.each(["medical-v1", "veterinary-v1"] as const)(
"should include domain=%s in connection URL",
async (domain) => {
Expand Down Expand Up @@ -597,6 +615,40 @@ describe("streaming", () => {
await connect(rt, server);
});

it("should include assemblyai-stt-3-7-realtime speech model in connection URL", async () => {
await cleanup();
WS.clean();

const wsUrl = `${websocketBaseUrl}?token=123&sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime`;
server = new WS(wsUrl);
rt = new StreamingTranscriber({
websocketBaseUrl,
token: "123",
sampleRate: 16_000,
speechModel: "assemblyai-stt-3-7-realtime" as const,
});
onOpen = jest.fn();
rt.on("open", onOpen);
await connect(rt, server);
});

it("should include assemblyai-stt-3-7-realtime-agent speech model in connection URL", async () => {
await cleanup();
WS.clean();

const wsUrl = `${websocketBaseUrl}?token=123&sample_rate=16000&speech_model=assemblyai-stt-3-7-realtime-agent`;
server = new WS(wsUrl);
rt = new StreamingTranscriber({
websocketBaseUrl,
token: "123",
sampleRate: 16_000,
speechModel: "assemblyai-stt-3-7-realtime-agent" as const,
});
onOpen = jest.fn();
rt.on("open", onOpen);
await connect(rt, server);
});

it("should parse speaker_label from turn event", async () => {
const turnPromise = new Promise<{ speaker_label?: string }>((resolve) => {
rt.on("turn", (event) => resolve(event));
Expand Down
Loading