Repository navigation
Count failed rounds, not calls, and ask at the failure limit - #153
Merged
Merged
Conversation
The failure cutoff counted each failing result, so a model that made three calls at once in one round was stopped by that round alone, and one trying three different queries a server refused was stopped before it could answer. It now counts as Cline's MistakeTracker does: a round in which every call failed counts once, and any call that works resets it. ReplToolLoopGuard counts the same way, so its one-round-short notice still lands on the round before the cutoff. At the cutoff, someone at the REPL is now asked whether to keep going, as Cline asks on its mistake limit: the full-screen REPL offers to keep going, keep going with a note for the model (read before its next call, as an interjection), or stop; the plain terminal asks y/N. With no one to ask (--yolo, the VS Code panel, input that isn't a terminal) the turn stops as before, then gets one reply with tools off, so it ends with the model saying what it tried and what went wrong rather than on a failed tool result.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The consecutive-failure cutoff counted each failing tool result and stopped the turn at 3, which cut off legitimate work:
This PR brings the cutoff in line with Cline's
MistakeTracker(sdk/packages/core/src/runtime/safety/mistake-tracker.ts).Count failed rounds, not calls
A round is the set of tool calls from one model reply. It counts once, and only if every call in it failed; any call that works resets the count.
ReplToolLoopGuardcounts the same way, usingFunctionCallIndex/FunctionCountto tell when a round has finished, so its "one more round will stop this turn" notice still lands one round before the cutoff. The default stays at 3 (repl.maxConsecutiveToolFailures): 3 rounds in which everything failed is about as lenient as Cline's 6 mistakes.Ask at the cutoff when someone can answer
As Cline does on its mistake limit, the REPL asks instead of stopping:
ReplInterjectingChatClientadds before its next call.Keep going? (y/N).When nobody can answer, stop and then reply once without tools
Under
--yolo, in the VS Code panel, or when input isn't a terminal, the turn stops as before. The model then gets one reply withToolMode.None, so the turn ends with it explaining what it tried, what went wrong and what to do next, instead of ending on a failed tool result. The tools stay declared for that reply, because some providers reject a history containing tool calls when no tools are declared. The prompt for that reply uses an internal correction prefix, so it isn't shown or replayed as user input.Testing
ToolMode.Noneand keeps the tools declared;--plain: answeringylet the turn finish.--yolo: the turn stopped, then the model gave its summary with no tools.