Skip to content

ci(stryker): set mutation score break threshold to 60 - #261

Merged
samtrion merged 1 commit into
mainfrom
fix/stryker-break-threshold-zero-pb
Nov 4, 2026
Merged

samtrion merged 1 commit into
mainfrom
fix/stryker-break-threshold-zero-pb

Conversation

@samtrion

@samtrion samtrion commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem

stryker-config.json has thresholds: high 80, low 60, break 0. Run 40488203974 (workflow_dispatch) ended with The final mutation score is 74.73 % and conclusion success, although the initial test run warned about failing tests (up to 30 in some warnings). With break 0 the workflow can never fail on score, so a regression is only visible on the dashboard.

Root cause

The break threshold was never set.

Change

thresholds.break: 0 -> 60 (equal to low; Stryker requires break <= low). The floor is about 15 points below the current 74.73% baseline because that baseline is depressed and noisy (failing initial tests).

Before / After

metric before after
thresholds.break 0 60
final score 74.73% 74.73% (unchanged, as expected)
workflow conclusion success success (break 60 not tripped): https://github.com/dailydevops/projectbuilders/actions/runs/40810530018
red path (throwaway branch, low=break=75) n/a failure, log Final mutation score is below threshold break. Crashing..., Stryker step failed, upload step skipped: https://github.com/dailydevops/projectbuilders/actions/runs/40815422940

Risk / what this does not change

  • A run with no score (0 mutants, coverage capture failure) does not trip break (NaN); it is not covered by this change.
  • When break trips, Stryker exits non-zero. In the shared pipeline the "Upload Stryker HTML reports" step has no if: always(), so the HTML artifact is skipped on such a run. The dashboard reporter runs inside Stryker and is unaffected.
  • Other repositories have the same setting and are handled separately.

thresholds.break was 0, so the mutation workflow stayed green regardless of
the score (74.73% in run 40488203974). Set it to the existing 'low' value (60),
about 15 points below the current baseline to absorb score swings caused by
failing initial tests. Stryker requires break <= low, which holds.
@codecov

codecov Bot commented Oct 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.69%. Comparing base (3375b9a) to head (baf403f).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #261   +/-   ##
=======================================
  Coverage   89.69%   89.69%           
=======================================
  Files          31       31           
  Lines         728      728           
  Branches       85       85           
=======================================
  Hits          653      653           
  Misses         67       67           
  Partials        8        8           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samtrion
samtrion merged commit 0d52e91 into main Nov 4, 2026
16 checks passed
@samtrion
samtrion deleted the fix/stryker-break-threshold-zero-pb branch November 4, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant