Symptom
make test reports FAIL github.com/mendixlabs/mxcli/mdl/linter with panic: test timed out after 10m0s, on a tree where the package passes when run on its own. It reads as a test failure in the PR under review, so a contributor stops and investigates their own change, and a reviewer reads a red suite.
Two contributors have measured it independently and reported it in their PR bodies, because nothing else in the repo records it:
| source |
mdl/linter wall time |
| #1274 |
~552s standalone — just under the limit |
| #1290 |
872s — over it |
In #1290 the contributor also checked that mdl/linter and mdl/catalog are byte-identical between the branch and its merge base, to establish it was not the PR.
Cause
make test is the only go test target in the Makefile with no -timeout:
test: grammar sync-all
CGO_ENABLED=0 go test ./...
so it takes Go's default of 10m per package. Every integration target sets one explicitly — test-integration uses -timeout 60m, the split suites -timeout 40m — so the unit-test target is the one left on the default, and it is now the one with a package close to it.
At ~552s the package is at 92% of the limit, which is why the outcome depends on load: under go test ./... parallelism it shares the machine with everything else, so it trips on a busy or slower runner and passes on an idle one. That is the worst failure mode to leave in place — it is not reproducible on demand, so each person who hits it re-derives the same diagnosis.
Suggested fix
Give make test an explicit -timeout, the way the integration targets have one. The number should have headroom over the slowest package rather than tracking it, since the point is that nobody should be reading a timeout as an assertion failure.
Worth considering alongside it, but separable:
- Why
mdl/linter takes ~9–15 minutes. It is far and away the slowest unit-test package; if that is a fixture being re-parsed per test rather than once, the timeout stops being interesting. Measuring -v run times would say.
- A standalone target (
make test-linter) or moving it to the integration set, if it is really integration-shaped work wearing a unit-test build tag.
Why file it
Both #1274 and #1290 carry a paragraph explaining this to whoever runs CI. That paragraph should not be the repo's record of it — the next contributor writes it again, and the next reviewer reads it again.
Symptom
make testreportsFAIL github.com/mendixlabs/mxcli/mdl/linterwithpanic: test timed out after 10m0s, on a tree where the package passes when run on its own. It reads as a test failure in the PR under review, so a contributor stops and investigates their own change, and a reviewer reads a red suite.Two contributors have measured it independently and reported it in their PR bodies, because nothing else in the repo records it:
mdl/linterwall timeIn #1290 the contributor also checked that
mdl/linterandmdl/catalogare byte-identical between the branch and its merge base, to establish it was not the PR.Cause
make testis the onlygo testtarget in the Makefile with no-timeout:so it takes Go's default of 10m per package. Every integration target sets one explicitly —
test-integrationuses-timeout 60m, the split suites-timeout 40m— so the unit-test target is the one left on the default, and it is now the one with a package close to it.At ~552s the package is at 92% of the limit, which is why the outcome depends on load: under
go test ./...parallelism it shares the machine with everything else, so it trips on a busy or slower runner and passes on an idle one. That is the worst failure mode to leave in place — it is not reproducible on demand, so each person who hits it re-derives the same diagnosis.Suggested fix
Give
make testan explicit-timeout, the way the integration targets have one. The number should have headroom over the slowest package rather than tracking it, since the point is that nobody should be reading a timeout as an assertion failure.Worth considering alongside it, but separable:
mdl/lintertakes ~9–15 minutes. It is far and away the slowest unit-test package; if that is a fixture being re-parsed per test rather than once, the timeout stops being interesting. Measuring-vrun times would say.make test-linter) or moving it to the integration set, if it is really integration-shaped work wearing a unit-test build tag.Why file it
Both #1274 and #1290 carry a paragraph explaining this to whoever runs CI. That paragraph should not be the repo's record of it — the next contributor writes it again, and the next reviewer reads it again.