Repository navigation
Conversation
Copy the CSV into the cache staging path so the original measurement remains available in the result bundle. The cached content is unchanged. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
BPF CI measures each veristat corpus in its own job and VM through the libbpf/ci selftest harness. Measuring several corpora in one VM needs results kept per corpus, independent of the selftest directory layout. Accept prepared object paths and checksums, and keep each corpus result in its own directory. Require valid, nonempty CSV before marking a measurement complete. Keep program rejection as data and continue after corpus errors so completed results remain available to the caller. Log input and failure context for CI diagnosis while keeping tool output in files. Keep the runner free of BPF CI specifics so it can move into a shared action if another project needs it. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Measuring corpora through the runner needs manifests of the selected objects and filters. Add a helper that expands per-corpus directory and glob selections into exact files and checksums, and define the kernel, Cilium, SCX, and Meta selections. Keep OSS and Meta input definitions separate so each caller selects its own dataset. Keep empty selections for the runner to report as missing required corpora. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Veristat currently runs through the libbpf/ci selftest harness and its per-corpus shell configs. run-vmtest also accepts a caller-supplied guest script through VMTEST_SCRIPT, sharing VM setup and status collection. Add an entrypoint that invokes the runner and records its aggregate exit status for the host. Per-corpus outcomes remain in the result files. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Successful tool exits can hide empty measurements, changed inputs, or missing completion receipts. Those cases can leave CI green while its comparison inputs are unusable or have the wrong object identity. Cover the CLI and receipt boundaries, and verify that the guest entrypoint records an incomplete measurement as a failing status. Run them in the existing unittest job. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
ci/veristat holds the runner, manifest helper, and BPF CI corpus selections. Move the baseline comparison script there too, so the veristat CI code lives in one place. No functional change. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
veristat_compare.py and its test are the last veristat CI code under .github. Move them next to the comparison script that runs the parser. Lint stops covering the parser once it leaves .github/scripts, and has never covered the other ci/veristat scripts. Run Black on the whole repository instead. The unit tests now live in one directory, so a single unittest step runs them. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
The kernel, Cilium, and SCX corpora each boot the candidate kernel in their own job through the libbpf/ci selftest harness. Prepare the three corpora once and measure them in one VM through the veristat entrypoint. Build SCX in the same job instead of passing it through an artifact. Keep the corpus baselines in one cache entry and compare the measured corpora in a loop, replacing the per-corpus steps of the local action, which the Meta workflow still uses. A push updates only the corpora it measured, so a failed corpus keeps its previous baseline as before. The new cache starts empty: comparisons report a missing baseline until the first push to each base branch. Upload the per-corpus CSV, stderr, and status files as one artifact. Drop the host-side verifier log dump, which only Cilium still enabled. It runs after the VM exits, on the runner's kernel, and looks for objects in the workspace root, so it cannot show the candidate kernel's logs. Kernel and SCX disabled it earlier for the same reason. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
The Meta workflow still runs veristat through the libbpf/ci selftest harness and the local comparison action. Prepare its manifest and measure it through the veristat entrypoint, with the same baseline steps as the OSS workflow under its own cache key. Publish only the result CSV, as before: Meta objects and raw tool logs must stay private. Remove the local comparison action and the legacy selector config, which have no remaining users. The new cache starts empty: comparisons report a missing baseline until the first push to each base branch. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Debugging a verification failure needs the verifier log from the kernel and objects that were measured. The removed host-side dump ran after the VM exited, so it could not provide it. After measuring a corpus, rerun veristat with -v for its rejected programs in the same VM and keep the output in failures.log, next to the CSV. The logs are diagnostics and do not change the measurement status. The OSS workflow already uploads the result directory; the Meta workflow publishes only the CSV, so Meta logs stay private. Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measure veristat corpora through a runner kept in
ci/veristat, booted by theexisting
libbpf/ci/run-vmtest@v4custom-script hook. No libbpf/ci change isneeded.
veristat-ossmeasures kernel, Cilium, and SCX in one VM, replacing threejobs.
veristat-metauses the same runner for the Meta corpus.changed, or empty results are errors, and one failing corpus does not hide
the others. Each corpus keeps its CSV, stderr, status, and, for rejected
programs, verifier logs captured in the VM.
workflow now keeps its baselines in one cache entry. The new key starts
empty: PRs report "No baseline" until the first push to each base branch.
workflows, and the legacy selector configs.
Local unit tests, ShellCheck, and a vmtest smoke pass; GitHub runs are untested.