Repository navigation
feat(roleplay): lead roleplay:auto with GLM 5.3, the best scorer in the chain - #99
Merged
Merged
Conversation
roleplay:auto now tries GLM 5.3 on OpenCode Go, ClinePass and NanoGPT before anything else. DeepSeek V4.1 Flash follows them, then GLM-5.3-Flash and GLM-5.2, and MiMo-V2.6-Pro is last. On 2026-10-29 three multi-turn roleplays with planted card rules, run twice each on OpenCode Go and scored out of 5, gave GLM 5.3 4.33 with no hard rule breaks (21-41 s a reply). DeepSeek V4.1 Flash, the previous lead, scored 3.25: it answered in 6-10 s but reasoned a tenth as much, broke a character rule and nearly confessed a secret. MiMo scored 2.83 and wrote a line for the user. The cost is OpenCode Go usage: a GLM 5.3 turn used about 30 times the allowance of a DeepSeek turn on 2026-10-25. The DeepSeek to GLM handoff is removed. A source hands off only to targets later in the chain, and every GLM 5.3 entry now comes before DeepSeek, so the handoff could never be offered; the DeepSeek leg would still have carried the tool with tool_choice "none".
Without ROLEPLAY_INTELLIGENCE_MODELS, roleplay:intelligence (and roleplay:auto under ROLEPLAY_AUTO_ROUTE=intelligence) now tries GLM-5.3 first, then GLM-5.3-Flash and GLM-5.2 on each provider, and MiMo-V2.6-Pro last. The roleplay:intelligence description served by /v1/roleplay/models says the same, so it no longer tells clients that MiMo leads. This matches production, where GLM 5.3 scored best of the chain's models in the 2026-10-29 roleplay test and MiMo scored worst.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
roleplay:autostarted every turn on DeepSeek V4.1 Flash because it was fast and cheap on the OpenCode Go allowance. In the 2026-10-29 roleplay test it wrote noticeably worse turns than GLM 5.3, so GLM 5.3 now goes first and becomes the recommended model forroleplay:auto.The test: three multi-turn roleplays with planted card rules, two runs each on OpenCode Go, every reply scored out of 5.
Change
ROLEPLAY_INTELLIGENCE_MODELS):opencode:glm-5.3,cline-pass:cline-pass/glm-5.3,nanogpt:z-ai/glm-5.3,opencode:deepseek-v4.1-flash,nanogpt:z-ai/glm-5.3-flash,nanogpt:z-ai/glm-5.2,cline-pass:cline-pass/glm-5.3-flash, then the three MiMo entries. The rest keeps its old relative order; MiMo moves to the end.roleplayHandoffTargets). Every GLM 5.3 entry now sits before DeepSeek, so the handoff could never be offered. DeepSeek fallback legs would still have carried the tool withtool_choice: "none"for nothing. The two keys leaveRUNTIME_FEATURE_FLAGS. They were never deployed as separate vars because feat(roleplay): let DeepSeek hand pivotal turns to GLM-5.3 #92's upload was rejected, sokeep_varscan't keep an old copy alive.ROLEPLAY_INTELLIGENCE_MODELSis unset) and theroleplay:intelligencedescription on/v1/roleplay/modelsno longer say MiMo leads. GLM-5.3 comes first there too.docs/roleplay.mdcites the test and the trade-off. On 2026-10-25 a GLM 5.3 turn used about 30× the OpenCode Go allowance of a DeepSeek turn, so Go's limits arrive much sooner. When Go refuses GLM 5.3, the turn falls to GLM 5.3 on ClinePass and NanoGPT, then DeepSeek.On 2026-10-29 only OpenCode answered. ClinePass returned 403
ENTITLEMENT_ERRORand NanoGPT 429model_cooldown. Both advance the chain and cool that entry for the session, so the order still holds.Verification
Vars only take effect on deploy.