Repository navigation
ci: persist golangci-lint analysis cache between runs - #11505
Conversation
|
|
||
| ## Tooling and Documentation | ||
|
|
||
| * CI now [persists `golangci-lint`'s per-file analysis |
There was a problem hiding this comment.
don't think we need release notes for this kind of changes
🟢 PR Severity: LOW
🟢 Low (2 files)
AnalysisThis PR only touches CI/CD workflow configuration ( To override, add a |
|
Addressed the review feedback in c99498b:
Verification: |
|
Correction to the previous commit (509f1c3): the nightly-only save gate was wrong for this branch's base. The Now: single |
|
Correction to my previous comment (and to 509f1c3): the claim that Fixed in 7140830: the exact key now ends in the run id ( |
7140830 to
bf3feb6
Compare
|
Rebased onto current master and changed in bf3feb6:
To verify after the nightly seeds the cache: a PR's lint log should show |
|
Fixed in b1b597a: the save step now runs after |
In this commit, we persist golangci-lint's per-file analysis cache in
the GitHub Actions cache store so the lint job stops re-typechecking the
whole repository from scratch on every run.
The lint targets bind mount ~/.cache/golangci-lint into the tools
container, but nothing persisted that directory between runs, so each job
started with an empty analysis cache ("File cache stats: 0 entries of
total size 0B" in the logs) and re-analyzed every package.
Every run restores the most recent snapshot, keyed by the lint config,
the tools module and the custom linter plugin sources, so a run never
reuses analysis results produced by a different linter. Only the nightly
master run saves a new snapshot: entries created by PR and merge queue
runs are only readable by that same PR and by master, and the repo's
shared cache space is already at its size limit. The save runs after
both lint steps and also when they failed, so the analysis that did run
isn't lost. A cancelled run saves nothing.
b1b597a to
0d5172b
Compare
|
Rebased onto master (now includes #11504) in 0d5172b. The branch is a single commit again: the restore step sits after the Buildx setup from #11504, and the nightly-gated save sits after |
Problem
The lint job re-analyzes the entire repository on every run because nothing persists
golangci-lint's per-file analysis cache between runs. Thelintandlint submodulessteps bind mount~/.cache/golangci-lintinto the tools container as/root/.cache/golangci-lint, but that directory is never saved or restored, so every job starts cold. The log confirms it: every run ends withFile cache stats: 0 entries of total size 0B, and the main-repo lint pass takes ~8 minutes (measured in job 126979734038, 01:46:55 to 01:55:12).The Go module cache and build cache are already restored through the
setup-gocomposite action, so dependency downloads aren't the issue here. The missing piece is the analysis cache that letsgolangci-lintskip packages whose sources haven't changed. This PR is separate from #11504, which caches the compiled linter binary itself (a ~3.6 minute saving); this one addresses the ~8 minute analysis pass.Solution
In this PR, we add an
actions/cachestep to thelintjob that restores and saves~/.cache/golangci-lint.The cache is keyed as follows:
lint-analysis-<os>-go-<version>-<config/tools hash>-<commit SHA>. The commit SHA makes the key unique per run so every run saves a fresh snapshot (a static key would save once and then never update).lint-analysis-<os>-go-<version>-<config/tools hash>-. A source-only change therefore restores the most recent previous snapshot and only re-analyzes the changed packages..golangci.yml,tools/go.mod,tools/go.sum,tools/.custom-gcl.yml, andtools/linters/**/*.go. Any change to the lint config, the pinned linter version, its dependencies, or the custom linter plugins falls back to a clean analysis rather than restoring results produced by a different linter.The existing cache cleanup job (deletes caches older than 12 hours) bounds how many snapshots accumulate, and GitHub's 10GB LRU eviction applies on top of that.
Expected impact
Locally, running the same image the CI lint job uses (
lnd-tools), a warm analysis cache cut the main-repo lint pass from 1m36s to 1m22s on a warm tree with Docker overhead dominating. The CI saving is larger because the CI pass is cold-typechecking everything withGOGC=50, which is what makes the cold 8 minutes. The exact CI saving can't be measured without a real runner round-trip; the expected effect is that packages whose sources didn't change are skipped.Verification
actionlintv1.7.12 on.github/workflows/main.yml: clean.lintjob now lists thegolangci-lint analysis cachestep before the lint steps.git diff --check: clean.scripts/check-release-notes.shwith the PR ref: passes.custom-gcl cache statusagainst the bind-mounted directory after a lint pass shows a populated cache (Size: 4.4MiB), and a second run is faster than the first.Cache restored from key: lint-analysis-...and aFile cache statsline with nonzero entries in the lint log.Out of scope
docker runinvocations. They share the same analysis cache directory, so submodule packages reused from the main pass benefit too, but a parallel fan-out or single-pass lint is a separate change.GOGC=50setting and the runner disk-cleanup step (~3 minutes) are untouched; whether they belong in the lint job is a separate question.