Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

## [4.43.3]

- `StreamingTranscriber` emits a `termination` event with the server's `Termination` message (`TerminationEvent`: `audio_duration_seconds`, `session_duration_seconds`), and `close()` resolves with it, or with `undefined` when it did not wait for the message or the message did not arrive before the timeout or the socket closed. `close()` now calls the `close` listener for a client-initiated close (code `1000`, reason `"Session ended"` or `"Closed by the client"`) before it clears the listeners; it is called once per session. Code that types `close()`'s result as `Promise<void>` must widen it to `Promise<TerminationEvent | undefined>`

## [4.43.2]

- `StreamingTranscriber.updateConfiguration()` takes `mode`, `max_turn_duration` and `continuous_partials`, which the streaming API accepts mid-stream, and takes each field by its camelCase connection-param name (`maxTurnSilence`, `keytermsPrompt`, ...) as well as its snake_case name, sending it as the snake_case field. The API ignored camelCase names before, so `updateConfiguration({ maxTurnSilence: 1500 })` from plain JavaScript changed nothing. When both names of a field are given with different values, the snake_case value is sent, with a warning. Unrecognized keys are still sent, with a warning. `format_turns` and `filter_profanity` are deprecated on `StreamingUpdateConfiguration` and warn when sent, since the API does not apply them mid-stream. New type `StreamingUpdateConfigurationParams`
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "assemblyai",
"version": "4.43.2",
"version": "4.43.3",
"description": "The AssemblyAI JavaScript SDK provides an easy-to-use interface for interacting with the AssemblyAI API, which supports async and real-time transcription and text-to-speech.",
"engines": {
"node": ">=18"
Expand Down
66 changes: 49 additions & 17 deletions src/services/streaming/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
WarningEvent,
HeartbeatEvent,
SilenceEvent,
TerminationEvent,
} from "../..";
import type { VadDetector, VadFrame } from "../../types/streaming/dual-channel";
import { EnergyVad } from "./energy-vad";
Expand Down Expand Up @@ -253,7 +254,9 @@ export class StreamingTranscriber {

private socket?: PolyfillWebSocket;
private listeners: StreamingListeners = {};
private sessionTerminatedResolve?: () => void;
private sessionTerminatedResolve?: (event?: TerminationEvent) => void;
/** True from `Begin` until the `close` listener has been called. */
private closePending = false;

// Dual-channel mode state (allocated only when params.channels is set).
private isDualChannel = false;
Expand Down Expand Up @@ -356,16 +359,24 @@ export class StreamingTranscriber {
}

/**
* Releases a `close()` that is waiting for the session to terminate.
* Releases a `close()` that is waiting for the session to terminate, with
* the `Termination` frame when there is one.
*
* Called from the `Termination` frame and from `onclose`. Without the
* `onclose` call, a socket that dies without a `Termination` frame leaves
* `close()` awaiting forever.
*/
private resolveSessionTermination(): void {
private resolveSessionTermination(event?: TerminationEvent): void {
const resolve = this.sessionTerminatedResolve;
this.sessionTerminatedResolve = undefined;
resolve?.();
resolve?.(event);
}

/** Calls the `close` listener once per opened session. */
private emitClose(code: number, reason: string): void {
if (!this.closePending) return;
this.closePending = false;
this.listeners.close?.(code, reason);
}

private connectionUrl(): URL {
Expand Down Expand Up @@ -640,6 +651,7 @@ export class StreamingTranscriber {
on(event: "heartbeat", listener: (event: HeartbeatEvent) => void): void;
on(event: "silence", listener: (event: SilenceEvent) => void): void;
on(event: "vad", listener: (event: VadFrame) => void): void;
on(event: "termination", listener: (event: TerminationEvent) => void): void;
on(event: "error", listener: (error: Error) => void): void;
on(event: "close", listener: (code: number, reason: string) => void): void;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
Expand Down Expand Up @@ -719,6 +731,7 @@ export class StreamingTranscriber {
if (settled) return;
settled = true;
if (timer) clearTimeout(timer);
this.closePending = true;
resolve(begin);
};

Expand Down Expand Up @@ -778,7 +791,7 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
// The socket is gone, so no `Termination` frame is coming. Release a
// `close()` that is waiting for one.
this.resolveSessionTermination();
this.listeners.close?.(code, reason);
this.emitClose(code, reason);
};

this.socket.onerror = (event: ErrorEvent) => {
Expand Down Expand Up @@ -870,7 +883,8 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
break;
}
case "Termination": {
this.resolveSessionTermination();
this.listeners.termination?.(message);
this.resolveSessionTermination(message);
break;
}
}
Expand Down Expand Up @@ -1195,22 +1209,25 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
this.send(JSON.stringify(message));
}

/** Awaits the session-termination frame, bounded by `timeoutMs`. */
/**
* Awaits the session-termination frame, bounded by `timeoutMs`. Resolves
* with the frame, or `undefined` when the timeout or a socket close comes
* first.
*/
private async waitForTermination(
sessionTerminated: Promise<void>,
sessionTerminated: Promise<TerminationEvent | undefined>,
timeoutMs: number,
): Promise<void> {
): Promise<TerminationEvent | undefined> {
if (timeoutMs <= 0) {
await sessionTerminated;
return;
return sessionTerminated;
}

let timer: ReturnType<typeof setTimeout> | undefined;
try {
await Promise.race([
return await Promise.race([
sessionTerminated,
new Promise<void>((resolve) => {
timer = setTimeout(resolve, timeoutMs);
new Promise<undefined>((resolve) => {
timer = setTimeout(() => resolve(undefined), timeoutMs);
}),
]);
} finally {
Expand All @@ -1233,11 +1250,17 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
* transcript and the session information.
* @param terminationTimeout - How long to wait for that frame, in
* milliseconds. `0` waits indefinitely. The socket closes either way.
* @returns The `Termination` frame (`audio_duration_seconds`,
* `session_duration_seconds`), or `undefined` when `close()` did not wait
* for it or it did not arrive before the timeout or the socket closed.
* The `close` listener is called before this resolves, unless the session
* never opened or the socket had already closed and reported it.
*/
async close(
waitForSessionTermination = true,
terminationTimeout = DEFAULT_TERMINATION_TIMEOUT_MS,
) {
): Promise<TerminationEvent | undefined> {
let termination: TerminationEvent | undefined;
if (this.flushTimer) {
clearInterval(this.flushTimer);
this.flushTimer = undefined;
Expand All @@ -1250,11 +1273,13 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
if (this.socket) {
if (this.socket.readyState === this.socket.OPEN) {
if (waitForSessionTermination) {
const sessionTerminatedPromise = new Promise<void>((resolve) => {
const sessionTerminatedPromise = new Promise<
TerminationEvent | undefined
>((resolve) => {
this.sessionTerminatedResolve = resolve;
});
this.socket.send(terminateSessionMessage);
await this.waitForTermination(
termination = await this.waitForTermination(
sessionTerminatedPromise,
terminationTimeout,
);
Expand All @@ -1270,9 +1295,16 @@ Learn more at https://github.com/AssemblyAI/assemblyai-node-sdk/blob/main/docs/c
this.socket.onerror = () => {};
}
this.socket.close();
// The socket's own close event is detached above, so report the
// client-initiated close here, before the listeners are cleared.
this.emitClose(
1000,
termination ? "Session ended" : "Closed by the client",
);
}

this.listeners = {};
this.socket = undefined;
return termination;
}
}
2 changes: 2 additions & 0 deletions src/types/streaming/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,7 @@ export type StreamingEvents =
| "heartbeat"
| "silence"
| "vad"
| "termination"
| "error";

export type StreamingListeners = {
Expand All @@ -220,6 +221,7 @@ export type StreamingListeners = {
heartbeat?: (event: HeartbeatEvent) => void;
silence?: (event: SilenceEvent) => void;
vad?: (event: VadFrame) => void;
termination?: (event: TerminationEvent) => void;
error?: (error: Error) => void;
};

Expand Down
112 changes: 112 additions & 0 deletions tests/unit/streaming-close.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,116 @@ describe("streaming close", () => {

await expect(rt.close()).resolves.toBeUndefined();
});

describe("termination and close events", () => {
const termination = {
type: "Termination",
audio_duration_seconds: 12,
session_duration_seconds: 14,
};

it("emits termination and resolves close() with the frame", async () => {
const onTermination = jest.fn();
const onClose = jest.fn();
rt.on("termination", onTermination);
rt.on("close", onClose);

const closePromise = rt.close();
await expect(server).toReceiveMessage(
JSON.stringify({ type: "Terminate" }),
);
server.send(JSON.stringify(termination));

await expect(closePromise).resolves.toEqual(termination);
expect(onTermination).toHaveBeenCalledTimes(1);
expect(onTermination).toHaveBeenCalledWith(termination);
expect(onClose).toHaveBeenCalledTimes(1);
expect(onClose).toHaveBeenCalledWith(1000, "Session ended");
// The termination listener runs before close() resolves, and close
// after it.
expect(onTermination.mock.invocationCallOrder[0]).toBeLessThan(
onClose.mock.invocationCallOrder[0],
);
});

it("emits termination for a frame the server sends unprompted", async () => {
const onTermination = jest.fn();
rt.on("termination", onTermination);

server.send(JSON.stringify(termination));
await new Promise((resolve) => setTimeout(resolve, 20));

expect(onTermination).toHaveBeenCalledWith(termination);
await rt.close(false);
});

it("calls close once when close() does not wait", async () => {
const onClose = jest.fn();
rt.on("close", onClose);

await expect(rt.close(false)).resolves.toBeUndefined();
expect(onClose).toHaveBeenCalledTimes(1);
expect(onClose).toHaveBeenCalledWith(1000, "Closed by the client");
});

it("calls close once and resolves undefined on timeout", async () => {
const onClose = jest.fn();
rt.on("close", onClose);

await expect(rt.close(true, 50)).resolves.toBeUndefined();
expect(onClose).toHaveBeenCalledTimes(1);
expect(onClose).toHaveBeenCalledWith(1000, "Closed by the client");
});

it("calls close once with the server's code when the server closes first", async () => {
const onClose = jest.fn();
rt.on("close", onClose);

const closePromise = rt.close();
server.close({ code: 1000, reason: "bye", wasClean: true });

await expect(closePromise).resolves.toBeUndefined();
expect(onClose).toHaveBeenCalledTimes(1);
expect(onClose).toHaveBeenCalledWith(1000, "bye");
});

it("does not call close again after a server-initiated close", async () => {
const onClose = jest.fn();
rt.on("close", onClose);

server.close({ code: 4031, reason: "idle", wasClean: false });
await new Promise((resolve) => setTimeout(resolve, 20));
await rt.close();

expect(onClose).toHaveBeenCalledTimes(1);
expect(onClose).toHaveBeenCalledWith(4031, "idle");
});

it("clears the listeners once closed", async () => {
const onClose = jest.fn();
rt.on("close", onClose);
await rt.close(false);

expect(
(rt as unknown as { listeners: Record<string, unknown> }).listeners,
).toEqual({});
// A second close() has no session to report.
await rt.close(false);
expect(onClose).toHaveBeenCalledTimes(1);
});

it("does not call close for a session that never opened", async () => {
const unopened = aai.streaming.transcriber({
websocketBaseUrl,
apiKey: "123",
sampleRate: 16_000,
});
const onClose = jest.fn();
unopened.on("close", onClose);

await expect(unopened.close()).resolves.toBeUndefined();
expect(onClose).not.toHaveBeenCalled();
await rt.close(false);
});
});
});
Loading