Repository navigation
Add example drawings to coding interviews - #262
Chuyutseng wants to merge 1 commit into
Conversation
46fe4ca to
ede0d50
Compare
afc217e to
9828618
Compare
|
After a reconnect in a coding interview the agent can be left holding code older than what the candidate typed, and if the interview ended while the drawing resend failed, the queued |
9d89db0 to
d6cea65
Compare
d6cea65 to
fa47c83
Compare
|
Could the example drawing area be something the candidate turns on in the lobby, off by default, rather than always present in a coding interview? CodeTrial aims to put the candidate in a situation as close to a real interview as possible, not to make the session as convenient as possible. A typical online coding interview gives the candidate a shared editor and nothing else, and explaining an example in words or as comments in the code is part of what is being practised. A drawing panel that is always there changes that setting by default, and it also changes what Jim sees (camera frames stop once a drawing arrives). The lobby already lets the candidate shape the interview (loop, Editor or Whiteboard), so an opt-in toggle there would fit. Someone who wants the aid can have it, and the default session stays as close to the real thing as possible. |
90aa7cf to
9bd5995
Compare
9bd5995 to
c91a8e9
Compare
c91a8e9 to
930a5b2
Compare
930a5b2 to
3a8c9f0
Compare
| .iter() | ||
| .any(|phrase| normalized.contains(&format!(" {phrase} "))); | ||
| if visual_reference || candidate_image { | ||
| errors.push(format!("{path}: example drawings are outside coding and communication assessment; remove the visual reference and any credit or deduction based on it. Use only code, tests or the candidate's actual conversation under the original rubric, never the interviewer's interpretation of a picture")); |
There was a problem hiding this comment.
This check can cost the whole report. Once drawing_received is set, normal coding advice like "Draw a diagram of the pointer moves before coding" in improvementPlan matches through recommendation, and "your sketch" matches through owned. After MAX_REPORT_REPAIRS, finish returns the error, because sanitize_report_candidate doesn't remove these strings and so there is never a salvage held. Make the drawing reference a salvage case instead: drop the offending item or sentence and accept the rest, rather than failing the report on wording.
|
Pen strokes in the example drawing carry a color object instead of a hex string, so every recorded stroke is rejected on replay and the color picker paints black. The detail is in the comment on |
3a8c9f0 to
4948e7f
Compare
|
@Chuyutseng Five |
4948e7f to
79cbfa3
Compare
ColtenOuO
left a comment
There was a problem hiding this comment.
The choice is not remembered. rememberSettings() and restoreSettings() in web/app.js save only the duration, the loop and the mode, so a candidate who turns drawing on finds the box unchecked again after a reload or on their next visit. The editor options on the same page are remembered through saveEditorOptions, so this one checkbox behaves differently from the others next to it.
Could you add it to the stored settings (written on change and restored in restoreSettings(), still unchecked when nothing is stored), and add a case to tests/browser/lobby.test.js that checks the box, reloads and expects it to stay checked?
79cbfa3 to
ca4e67d
Compare
jserv
left a comment
There was a problem hiding this comment.
Rebase latest main branch and resolve conflicts.
ca4e67d to
8c67c64
Compare
Let candidates explain examples without hiding their code. Offer pen, eraser, undo and clear in an optional drawing area. Send example images only when Jim requests read_board. Preserve the camera policy and keep drawings outside assessment. Save drawings in reports and replay as session artifacts. Closes sysprog21#255
8c67c64 to
7213b57
Compare
Coding candidates need a way to draw examples while keeping their code visible. This adds an optional, collapsible Example drawing area below the editor, beside the candidate test cases. It is disabled by default in the lobby. The lower panel stays compact when collapsed, and candidates can drag the divider to resize it.
The first version includes pen, eraser, undo and clear, with color selection. The buttons use icons with accessible labels and tooltips. Shapes will be added in a later change.
Drawings are sent to Gemini Live only when requested. The browser keeps the latest JPEG snapshot on the server, but edits, drawing handovers and reconnects do not send it to Gemini. When the candidate refers to the drawing, Jim is instructed to call
read_board. The server then sends the latest image throughrealtimeInput.videobefore returning the tool response, at most once per tool batch. A drawing that is not referenced adds no image inputs. Once read, the image remains in the Live context and incurs the existing per-turn context cost.Drawing activity delays idle reminders, and stopping drawing can hand the turn back to Jim without automatically sending an image. Code and drawings are preserved in recordings and replay, and the final drawing is shown separately in the report. Drawings do not add or deduct coding or communication points and are not sent as images to the report model.
Validation:
./scripts/test.shexited with status 0, with all 1118 browser tests passing.Closes #255
Summary by cubic
Adds an opt-in Example drawing area to coding interviews so candidates can sketch examples without hiding their code. The lobby toggle is disabled by default and unavailable in Whiteboard, and the choice is signed into token metadata, so disabled or non-coding sessions refuse drawing streams and
read_boardcalls. Drawings stay out of assessment, and the contract bundle is bumped to 35.read_board, once per turn.Closes #255.
Written for commit 7213b57. Summary will update on new commits.