Repository navigation
fix(driver): keep USB transfers within Linux's usbfs memory pool - #70
Merged
Merged
Conversation
…SB memory Linux charges the in-flight transfers of every program to one pool, usbcore's usbfs_memory_mb (16 MiB by default). A transfer that does not fit fails with ENOMEM, which nusb passes on as an unknown error, so the interface showed "USB communication failed: usb transfer: unknown (errno 12)" and gave no hint what to do. Map ENOMEM on Linux to a new UsbError::NoMemory, so drivers can tell a refused transfer from a failed one, and show error.device.usbMemory, which names the usbfs pool to raise. Other systems keep their codes: 12 means something else there.
On Linux every SLogic capture at a high sample rate failed after exactly 15 MiB of data with "usb transfer: unknown (errno 12)". The driver queued sixteen 1 MiB requests, and Linux charges in-flight transfers to usbcore's usbfs pool, 16 MiB by default and shared by every program: the sixteenth request, plus the kernel's bookkeeping, no longer fit. macOS and Windows have no such pool. Sipeed's driver hit the same limit and changed its ring in libsigrok slogic-dev ba92c5fe: requests of about 5 ms of data, within [32 KiB, 2 MiB], and at least four in flight, halving the requests when usbfs refuses them. Follow that sizing, keeping 1 MiB as the largest request (nusb always reads with WinUSB RAW_IO), and plan the ring within a budget instead of probing: nusb reports a refused submission only in its turn, after RUN, so it cannot be found and halved before the capture starts. UsbTransport::transfer_budget gives that budget: on Linux three quarters of the pool read from /sys/module/usbcore/parameters/ usbfs_memory_mb (12 MiB by default, none when the pool is unlimited), elsewhere none. The ring holds what fits, twelve 1 MiB requests by default, and requests shrink until four fit in a smaller pool. Mock devices can set a budget for tests.
…fers The budget leaves a quarter of usbfs's pool to other programs, but one that holds more can still make Linux refuse an SLogic request. Such a request never reached the device, so no data is missing, yet the capture failed on it. Sipeed's driver keeps the requests it could submit as long as four are in flight, since with fewer the analyzer's FIFO overflows and the firmware drops data unreported. Do the same when nusb reports the refusal in the request's turn: leave the request out of the ring, also while re-arming, and only fail, with the usbfs hint, once fewer than four would remain.
Buffer mode uploads on USB 3.0 models queue sixteen 1 MiB transfers, the numbers DSView uses on Windows (it keeps one in flight elsewhere). That overflows Linux's default 16 MiB usbfs pool, so the sixteenth transfer failed and the upload stopped after 15 MiB. Stream mode at the highest rates could come close as well. Keep both modes within the transport's transfer budget: twelve 1 MiB uploads under the default pool, unchanged elsewhere.
At the fastest USB 3.0 rates the driver keeps four 4 MiB blocks in flight, read as sixteen 1 MiB transfers, which overflows Linux's default 16 MiB usbfs pool: the sixteenth transfer failed. Keep the transfers within the transport's transfer budget: twelve under the default pool, three blocks' worth, unchanged elsewhere.
Tell users that Linux shares one pool of USB transfer memory between programs, that HSync keeps within three quarters of it, and what to do when a capture still stops for lack of it. The SLogic crate's README records the later Sipeed revision its transfer ring follows and how the ring departs from it.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Linux, SLogic captures at high sample rates failed after exactly 15 MiB with
usb transfer: unknown (errno 12), while macOS was fine. Linux charges the in-flight transfers of every program to one pool, usbcore'susbfs_memory_mb(16 MiB by default). The SLogic driver queued sixteen 1 MiB requests, and the kernel's bookkeeping on top of the sixteenth no longer fit. nusb tried a zero-copy buffer first (theFailed to allocate zero-copy bufferwarning), fell back to a heap buffer, and the submission failed with ENOMEM. nusb reports a failed submission in its turn, so the first 15 requests delivered their data and the capture failed on the sixteenth. The sample counts in the report match: 7,864,256 samples × 2 bytes and 15,728,576 × 1 byte are both 15 MiB, less the head bytes. DSLogic USB 3.0 Buffer mode (16 × 1 MiB) and PXLogic at its fastest USB 3.0 rates (four 4 MiB blocks) had the same problem.Sipeed hit this in its own driver and reworked the transfer ring in libslogic 2.3.3 (sipeed/libsigrok
slogic-devba92c5fe, sipeed/SLogic f06492f):This PR follows that sizing. It plans the ring to fit instead of probing, because nusb reports a refused submission only in its turn, after RUN.
UsbError::NoMemory: Linux's ENOMEM gets its own variant, and the error keyerror.device.usbMemorytells the user to raise/sys/module/usbcore/parameters/usbfs_memory_mbinstead of showing errno 12.UsbTransport::transfer_budget: on Linux, three quarters of the pool, read from sysfs. That is 12 MiB by default, and no limit when the pool is set to 0 (unlimited). Other systems have no budget. Mock devices can set a budget for tests.Validation:
cargo clippy --workspace --all-targets -- -D warningscargo testfor hsync-driver, hsync-slogic, hsync-dslogic, hsync-pxlogic, hsync-atklogic and hsync-session: 280 passed. New cases cover: the budget arithmetic; the ENOMEM mapping (the positive case runs only on Linux, i.e. in CI); SLogic plans under the default and small pools; the ring the transport budget allows; captures that go on through refused requests; and the usbfs hint when fewer than four requests remain.pnpm test: 86 files, 609 tests passed, including the locale key parity