Repository navigation
cpu: recognise IRIX 6.5's idle loop and park it - #114
Merged
Merged
Conversation
Author
IRIX 6.5 never parked. `idleprof` at a settled IP22 getty shows why: the idle path is a ~470-byte scheduler loop (0x880daedc..0x880db0b4, ~89 instructions) with calls to spl and the ll/sc lock and stores to kernel globals. Its registers evolve every iteration, so the old PC+GPR hash never repeated. The same hash *did* match short device-poll loops (lbu/andi/bne -3 waiting on a SCSI bit), which must keep executing, so the detector was wrong in both directions. Replace it with a PC-cluster sample. The run loop samples the PC every 32 instructions into a ring of 32 (~1000 instructions); samples are sorted into clusters (consecutive PCs <=256 bytes apart), and the CPU parks when one kernel cluster holds at least 12 samples with 12 distinct PCs. That rejects 2-instruction DELAY loops, 3-instruction device polls, the 8-instruction ll/sc lock spin and mixed work, while accepting 5.3's 14-instruction loop and 6.5's 89-instruction scheduler path. User-mode samples are tolerated up to a quarter of the ring (a syscall wake runs a few user instructions between kernel stretches) but a user-dominated ring is never parked. All earlier safety rules stay: no parking with IE=0 or every mask clear, wake on any pending line and on running=false, the masked-wake back-off, and no park before Compare is armed. A 60 s warm-up from process start (IRIS_IDLE_WARMUP_SECS overrides) keeps boot at full speed: parking is a steady-state optimisation and IRIX boot runs long instruction-counted waits. The runtime switch and counters are the monitor's `idle on|off|status` and `idleprof stats`. Measured on callisto (x86_64, 16 cores), IRIX 6.5 IP22, R4400, fresh disk: - settled getty: 2 %, 2 %, 2 % host CPU (was ~200 %). - two parallel instances, each getty: 1-2 % each, both responsive. - cold boot to login: 42 s (was >400 s). - ci_clock,idle-pause cold boot to login: 46 s. IRIX 5.3 was not verified: the rig has no 5.3 image. Its 14-instruction loop fits the new rules and the park/wake mechanics are unchanged.
The seeq-enet loop sets `busy = tx_moved || rx_moved` at the end of a pass and skips its wait while busy, so a burst is pumped without pacing. The "nothing moved" fast path `continue`d before that assignment, so after the first frame that was sent or delivered `busy` stayed true forever and the thread spun at 100 % for the rest of the run. Clear it before the `continue`. This was the other half of an idle IRIX 6.5 instance's ~200 % host CPU. With the CPU idle detector fixed and this, two parallel 6.5 instances at a getty use 1-2 % each. `top -H` on callisto, 6.5 IP22 at a getty: seeq-enet ~99 % -> ~0 % (sleeping).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
IRIX 6.5's idle loop is now recognised and parked. A settled 6.5 IP22 instance at a getty uses ~2 % host CPU per instance (1–2 % for two in parallel) instead of ~200 %, and the wall-clock
idle-pausecold boot is back to 42–50 s (was >400 s). Related: #109 (the old detector could park the no-media PROM boot).The change has two parts:
src/cpu/idle_park.rs,mips_exec.rs): replace the exact PC+GPR hash with a PC-cluster sample; add a 60 s boot warm-up; addidle on|off|statusandidleprof stats.Capture (the reason for the rewrite)
idleprofon a settled 6.5 IP22 getty (R4400, fresh disk, headless, serial console, xdm/amstart killed) shows the idle path is not a tight branch:splat0x8801_03xx, the ll/sc lock at0x8801_ba90, and writes kernel globals.lbu/andi/bne -3, waiting on a SCSI bit at0x8805604c), which must keep executing. That is why the wall-clockidle-pauseboot stalled: parking a poll loop that needs instructions.clanker/perf/idle-pause-work.md) is the middle case.The detector
The batch run loop calls the sampler after each ten-instruction group; with
SAMPLE_EVERY = 32that is ~320 guest instructions per record and ~10,000 per 32-sample ring. (The single-step path records every 32 calls.)updatesorts the samples and groups them into clusters (consecutive PCs ≤256 bytes apart), then parks when:IRIS_IDLE_WARMUP_SECSoverrides, 0 disables). Parking is a steady-state optimisation; boot has long instruction-counted waits that must not be sliced. The timer is process-wide so monitor stop/start does not restart it.idle statusreports park count, sleep time, wake reasons and the last window's largest cluster.Evidence
Host:
callisto, x86_64, 6 cores (i5-8500T @ 2.1 GHz). Guest: IRIX 6.5 IP22, R4400, fresh sparse copy of the rig disk per run,nvram-rig.bin, telnet forwarded, headless;killall amstart; killall xdmafter boot, then settle.idle-pause(3 × 10 s)dateidle-pausecold boot to loginci_clock,idle-pausecold boot to logindate/sleep 5across a parkcpu stop; cpu startThe "before" 194–202 % is CPU thread ~100 % plus
seeq-enet~100 %.top -Hbefore the seeq fix:seeq-enet98.9 %,MIPS-CPU28 % (parked),hptimer15.6 %. After:seeq-enetsleeping, total 1–2 %.Tests on a quiet host:
cargo test --workspacejitv2cargo test --workspace --features jitv2ci_clockcargo test --workspace --features ci_clockidle-pausecargo test -p iris --lib --features idle-pauseci_clock,idle-pausecargo test -p iris --lib --features ci_clock,idle-pauseNew tests:
update_parks_a_long_stationary_kernel_window,update_parks_a_dominant_cluster_with_distant_helper_calls,update_declines_a_short_device_poll_loop,update_declines_a_two_instruction_delay_loop,update_declines_a_user_mode_loop,update_declines_a_window_that_keeps_moving, plus the existing wake/park/pace tests. Themips_exec_testmasked-interrupt park test now samples as the run loop does and uses a loop length coprime with the batch.Not verified / remaining risks
save/restore(the pre-existing restore-fidelity issue inclanker/perf/idle-pause-work.md§4.2); the save itself works and the CPU parks correctly afterrestorein the cases where the guest survives. Not caused by this change.iris-ci runtimed out waiting for its marker); telnet over the forwarded port was used for all guest commands. The serial getty is present and the boot serial log is readable.IRIS_IDLE_WARMUP_SECS=0andIRIS_NO_IDLE=1are the escape hatches.Relates to #109.