I measure where the bottleneck actually is, on hardware I can put an ammeter on, and I publish the data whether or not it agrees with me.
Website · ORCID · Google Scholar
Electrical and computer engineer (UMass Amherst), based in Austin. Hardware engineer at Dell by day, on enterprise AI-server platforms, GPU and PCIe subsystems, and root-cause work. The research below is my own, run on my own hardware.
llama-roofline · zenodo.21842493
Is your llama.cpp decode memory-bandwidth-bound? Measure it in one command. Prints the operating point beside every fitted bandwidth, and refuses to quote a number for silicon it has not measured. Docs: manunicholasjacob.com/llama-roofline, with an in-browser GGUF inspector.
ml-systems-lab · zenodo.21867055
One YAML config drives llama.cpp and ONNX Runtime sweeps across laptops, a Raspberry Pi over SSH, and GPUs. TTFT, throughput, and per-rail energy from the same run.
qgemv-roofline · zenodo.22164007
Twelve CUDA kernels and a Triton implementation of the batch-1 decode GEMV over ggml q8_0 and q4_0, scored against a bandwidth ceiling the harness measures on the same device rather than reads off a spec sheet. Five NVIDIA GPUs, sm_60 through sm_89.
Each one ships its data, its harness, and a DOI.
The memory wall at the edge of language · zenodo.21844855
Edge LLM decode is bandwidth-bound, with one roofline fitting all seven measured configurations at R² = 0.98 on Arm and on x86, and the KV cache hits a capacity wall well before compute does.
The break-even parallel speedup · zenodo.21987261
Multithreading saves energy exactly when parallel speedup beats the power ratio. Measuring the static power floor at every operating point shows the answer turns on whether the board's idle power is charged to the inference.
Your quantization format is not free · zenodo.21938812
What a GGUF label does and does not tell you about the file: at small model sizes the label is not the bit rate, and the community-default format pays a measurable cost on every core tested.
The cold-start tax · zenodo.21844857
What a duty-cycled wake actually costs, and the eviction cliff sitting behind it.
gguf-faultscope · zenodo.22163327
What one flipped bit does to a quantized model, and where in the file it has to land before anything downstream notices.
NVIDIA/garak · three merged
A config_root fix in goodside.RileyIsnt, an ASCII-selection guard in badchars, and a restored unreachable-error path in load_plugin. The first shipped in garak v0.16.0.
ai-dynamo/aiperf · four merged, more open
Merged: #1340, a CLI test that asserted across a Rich line wrap; #1350, which probes the NVML energy counter once instead of failing silently; #1320, which makes crick optional so pip install aiperf needs no C toolchain on aarch64; and #1470, which keeps log lines a Windows console cannot encode. Open: a host-telemetry collector category with a RAPL backend, because a CPU-only run currently reports no energy at all, and several fixes for defects found by building against the code rather than reading it.
Arm Learning Paths: one merged
#3848, a Learning Path on measuring whether LLM decode is memory-bound on an Arm CPU: the board's read ceiling, a llama.cpp sweep over model sizes and thread counts, and a llama-roofline diagnosis, with every output block taken from one Raspberry Pi 5 capture run. Merged 12 October 2026; not yet live on learn.arm.com.
ggml-org/llama.cpp and vllm-project/vllm · issues
Measurement-backed bug reports, including an effective-bandwidth characterisation of Arm decode. Engineers with different hardware have reproduced several of them in-thread, on a 5090 and on server parts.
Journal of Open Source Software · reviewer
Reviewing submissions, which so far has meant finding a sign error in a published MALA sampler and six merged fixes to the package under review. One reviewed paper is published: doi:10.21105/joss.11187.
ATC '26 Artifact Evaluation Committee · member
Two artifact reviews delivered, on 27 September and 2 October 2026.