Skip to content

[OMEGA-490] Out of token budget notification fix - #413

Merged
TossSky merged 4 commits into
patch-to-v0.1.20from
omega-490-fix
Oct 27, 2026
Merged

TossSky merged 4 commits into
patch-to-v0.1.20from
omega-490-fix

Conversation

@paul-v-snet

@paul-v-snet paul-v-snet commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Description

  • The antispam filter has been restored since it turned out that it's still necessary for small models.
  • The antispam filter description has also been restored.
  • A separate expression, notify, has been implemented in the channels - same as send, but without the antispam filter and without updating lastsend. It serves only for harness notices, not for LLM answers.
  • Autotests have been updated accordingly.

The main reason for implementing a separate expression for harness notices is that in a previous Omega version, we discovered an issue where a notice was sent only once because the antispam filter blocked repeated messages.

For instance, a user sends a request, then an LLM returns a response with incomplete_reason=max_output_tokens, and the harness sends a notice via the send expression.
If the user sends another request and an LLM returns a response with incomplete_reason=max_output_tokens again, the notice is blocked by the antispam filter, and the harness doesn't send anything. So that notice could only be sent once until the LLM returned a complete response.

During an internal team discussion, we came to the conclusion that this filter was no longer relevant. So we decided to remove it, and this automatically solved the issue described above in #383.

But during QA, we found that small models were still spamming without any new user input, so the antispam filter is still necessary in Omega. And that's the reason for restoring it.

But in that case, send won't work for repeated notices because the antispam filter will discard them if the previous message was also a notice. So implementing a separate expression for harness notices was necessary, and that's exactly what has been done in this PR.

The previous logic for managing incomplete LLM responses, suggested by @TossSky, still remains in the providers. But notice sending has been moved to the main loop. Now each LLM response is checked for LLM_OUTPUT_LIMIT_MARKER, and if the marker is found, the notice is sent via notify, while send sends nothing.

How Has This Been Tested?

Live ran + mock/unit - all passed/green.

Checklist

  • PR contains autogenerated code
  • Self-review completed
  • Test scenarios above are passed with the version of the code from PR

@TossSky

TossSky commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Tested: PR #413 at c2b127d, image built from that commit (Omega version=v0.1.22-4-gc2b127d), started with scripts/omega from the branch. Compared with v0.1.22 and v0.1.20 in the same setup.

What I checked

  • run_mandatory 178 passed, run_optional 6 passed, tests/pytest.sh 135 passed, Autotests/unit/test_llm_budget.py 27 passed
  • Repeated messages are dropped again (src/channels.metta:24-29). With ASICloud minimax-m3 over WebSocket, 10 questions with 4 idle minutes after each answer were followed by 0 to 7 extra messages, none of them identical to the previous one. On v0.1.22 the same setup gave up to 30 extra messages per answer, with the same text up to 17 times in a row
  • The limit notice reaches the user for every message (src/loop.metta:97-98, src/channels.metta:32). With OpenAI gpt-5.5 and maxOutputToken: 16, each of three user messages got exactly one notice, although the text was the same each time, and 150 idle seconds brought none
  • notify is not among the commands the parser accepts from the model, so the model cannot bypass the filter with it

Note

On builds with the filter, minimax-m3 sometimes leaves a question unanswered and keeps sending "No new input to respond to." instead, which the filter drops. 3 of 10 questions went unanswered on this PR and 3 of 5 on v0.1.20, while v0.1.22 without the filter answered 7 of 7. The cause is not established, and the behavior is the same as in v0.1.20

Verdict: PASS

@TossSky
TossSky merged commit 970dd4a into patch-to-v0.1.20 Oct 27, 2026
4 checks passed
@paul-v-snet paul-v-snet changed the title [OMEGA-490] Out of token budget notice fix [OMEGA-490] Out of token budget notification fix Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants