[yanyucai]

Summary

During long-form transcription on audio with prolonged silence, music, or low-energy speech, Whisper's autoregressive decoder can get trapped in an attractor repetition loop:

1. On silent or low-signal chunks, no_speech_prob often fails to reach the 0.6 threshold, producing a hallucinated segment (e.g. repeated common phrases).
2. Because last_ts_pos for the short hallucinated phrase is brief (often 1–2 seconds), seek only advances by that small interval.
3. Crucially, the hallucinated segment tokens are appended to prev_chunk_segments and prev_history_segments.
4. In subsequent 30s windows, the decoder is conditioned on this prefix via <|startofprev|>. Since the background remains quiet, the model attends back to the prompt, outputting the exact same text again and again in an unbroken autoregressive loop across thousands of chunks.

Changes

• Early repetition termination: In whisper_run's step loop, call transcribe::stop_on_repetition on generated_text_ids to catch intra-chunk repetitive token blocks early (aligning with other families in the codebase such as MOSS and Granite).
• Inter-chunk duplicate suppression: Detect consecutive identical or near-identical segments within close temporal proximity (<= 15s). When detected:

• Drop the repetitive hallucinated segments from the emitted results.
• Clear prev_chunk_segments and prev_history_segments.
• Reset do_condition_on_prev_tokens = false to break the feedback chain.
• Advance seek by the full chunk window (seek_num_frames) to jump past the silent attractor region.

AI Assistance

I used an AI coding assistant to help draft and refine the C++ implementation for the repetition guard and context clearing logic. I reviewed the code diff and performed end-to-end transcription testing locally.

Validation

• Rebuilt universal dylib on macOS (AppleClang 17, Metal + CPU).
• Tested with long-form audio containing silence and music sections using ggml-large-v3-turbo-q5_0.bin:

• Before: Looped on identical hallucinated sentences across consecutive windows, locking the sliding seek window to small increments.
• After: Repetitive generation loop is caught early, duplicated segments are suppressed, context history is cleared, and subsequent genuine dialogue decodes cleanly with accurate timestamps.