Skip to content

ci: disable the mutation tests - #916

Merged
vbreuss merged 1 commit into
mainfrom
ci/disable-mutation-tests
Oct 29, 2026
Merged

vbreuss merged 1 commit into
mainfrom
ci/disable-mutation-tests

Conversation

@vbreuss

@vbreuss vbreuss commented Oct 29, 2026

Copy link
Copy Markdown
Member

Since the migration to TUnit, Stryker runs the tests through its runner for the Microsoft.Testing.Platform, which leaves mutants pending. The mutation dashboard rejects such reports, so every build on main has failed and a tag would not be published. Same as in aweXpect, the mutation tests stay disabled until Stryker fixes the runner.

Since the migration to TUnit, Stryker runs the tests through its runner for the Microsoft.Testing.Platform, which leaves mutants pending. The mutation dashboard rejects such reports, so every build on main has failed and a tag would not be published. Same as in aweXpect, the mutation tests stay disabled until Stryker fixes the runner.
@vbreuss vbreuss self-assigned this Oct 29, 2026
Copilot AI balanced review requested due to automatic review settings October 29, 2026 23:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@vbreuss
vbreuss merged commit 5466462 into main Oct 29, 2026
17 checks passed
@vbreuss
vbreuss deleted the ci/disable-mutation-tests branch October 29, 2026 23:21
@sonarqubecloud

Copy link
Copy Markdown

github-actions Bot added a commit that referenced this pull request Oct 29, 2026
github-actions Bot added a commit that referenced this pull request Oct 29, 2026
@github-actions

Copy link
Copy Markdown

🚀 Benchmark Results

Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
Intel Xeon Platinum 8370C CPU 2.80GHz (Max: 3.42GHz), 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Method N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 270.8 ns 2.96 ns 2.47 ns 0.63 1976 B 1.00
Mockolate 1 426.8 ns 11.01 ns 10.30 ns 1.00 1976 B 1.00
Imposter 1 249.8 ns 3.47 ns 3.24 ns 0.59 928 B 0.47
TUnitMocks 1 576.4 ns 9.23 ns 8.18 ns 1.35 2088 B 1.06
Moq 1 136,957.5 ns 992.35 ns 928.25 ns 321.08 14938 B 7.56
NSubstitute 1 6,526.6 ns 17.72 ns 16.58 ns 15.30 9280 B 4.70
FakeItEasy 1 5,857.4 ns 31.62 ns 29.57 ns 13.73 8251 B 4.18
baseline* 10 496.7 ns 13.62 ns 12.74 ns 0.58 2192 B 1.00
Mockolate 10 855.0 ns 6.80 ns 6.36 ns 1.00 2192 B 1.00
Imposter 10 581.6 ns 5.32 ns 4.97 ns 0.68 1792 B 0.82
TUnitMocks 10 1,758.9 ns 9.27 ns 8.67 ns 2.06 3840 B 1.75
Moq 10 141,792.5 ns 354.38 ns 314.15 ns 165.84 18930 B 8.64
NSubstitute 10 9,299.7 ns 22.13 ns 19.62 ns 10.88 11800 B 5.38
FakeItEasy 10 9,415.7 ns 78.48 ns 65.54 ns 11.01 15786 B 7.20
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
AMD EPYC 7763 2.78GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

CreateMock Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 20.24 ns 0.276 ns 0.258 ns 1.17 160 B 1.00
Mockolate 17.32 ns 0.098 ns 0.076 ns 1.00 160 B 1.00
Imposter 113.22 ns 1.107 ns 1.036 ns 6.54 576 B 3.60
TUnitMocks 40.71 ns 0.393 ns 0.348 ns 2.35 216 B 1.35
Moq 1,420.38 ns 8.332 ns 7.794 ns 82.01 2096 B 13.10
NSubstitute 1,935.16 ns 8.681 ns 8.120 ns 111.74 5048 B 31.55
FakeItEasy 1,842.71 ns 27.354 ns 22.842 ns 106.40 2763 B 17.27
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
INTEL XEON PLATINUM 8573C 3.02GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Event Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 310.9 ns 9.25 ns 8.65 ns 0.89 1744 B 1.00
Mockolate 349.6 ns 7.86 ns 7.36 ns 1.00 1744 B 1.00
Imposter 200.3 ns 1.35 ns 1.27 ns 0.57 512 B 0.29
TUnitMocks 233.5 ns 5.58 ns 5.22 ns 0.67 1392 B 0.80
Moq 13,685.0 ns 99.09 ns 92.69 ns 39.16 12809 B 7.34
NSubstitute 5,026.8 ns 23.68 ns 20.99 ns 14.38 9264 B 5.31
FakeItEasy 117,315.1 ns 1,322.05 ns 1,171.96 ns 335.67 15715 B 9.01
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
AMD EPYC 7763 2.58GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Property N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 328.3 ns 5.68 ns 5.31 ns 0.62 2472 B 1.00
Mockolate 1 530.5 ns 14.73 ns 13.78 ns 1.00 2472 B 1.00
Imposter 1 181.1 ns 1.54 ns 1.37 ns 0.34 664 B 0.27
TUnitMocks 1 457.7 ns 4.10 ns 3.83 ns 0.86 1696 B 0.69
Moq 1 11,921.7 ns 40.81 ns 36.18 ns 22.49 10641 B 4.30
NSubstitute 1 7,874.2 ns 33.90 ns 31.71 ns 14.85 11720 B 4.74
FakeItEasy 1 8,518.3 ns 64.56 ns 60.39 ns 16.07 11508 B 4.66
baseline* 10 794.2 ns 4.16 ns 3.47 ns 0.78 2976 B 1.00
Mockolate 10 1,016.1 ns 8.09 ns 7.57 ns 1.00 2976 B 1.00
Imposter 10 348.8 ns 1.69 ns 1.58 ns 0.34 952 B 0.32
TUnitMocks 10 1,658.4 ns 8.56 ns 7.59 ns 1.63 4048 B 1.36
Moq 10 18,823.7 ns 47.70 ns 37.24 ns 18.53 18721 B 6.29
NSubstitute 10 18,063.8 ns 62.00 ns 51.77 ns 17.78 21585 B 7.25
FakeItEasy 10 19,956.0 ns 104.18 ns 92.35 ns 19.64 31546 B 10.60
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
AMD EPYC 7763 3.05GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v3

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Callback Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 675.4 ns 8.29 ns 7.75 ns 1.86 1608 B 1.00
Mockolate 363.9 ns 11.87 ns 11.11 ns 1.00 1608 B 1.00
Imposter 250.8 ns 2.86 ns 2.38 ns 0.69 744 B 0.46
TUnitMocks 549.0 ns 7.95 ns 7.05 ns 1.51 2064 B 1.28
Moq 99,691.2 ns 414.32 ns 367.28 ns 274.20 9176 B 5.71
NSubstitute 4,889.9 ns 33.80 ns 31.62 ns 13.45 7896 B 4.91
FakeItEasy 4,992.0 ns 73.49 ns 68.75 ns 13.73 6970 B 4.33
Details

BenchmarkDotNet v0.15.8, Linux Ubuntu 26.04.1 LTS (Resolute Raccoon)
AMD EPYC 9V45 4.42GHz, 1 CPU, 4 logical and 2 physical cores
.NET SDK 10.0.303
[Host] : .NET 10.0.14 (10.0.14, 10.0.1426.52326), X64 RyuJIT x86-64-v4

Job=InProcess Toolchain=InProcessEmitToolchain IterationCount=15
LaunchCount=1 WarmupCount=10

Indexer N Mean Error StdDev Ratio Allocated Alloc Ratio
baseline* 1 887.8 ns 7.54 ns 7.05 ns 1.74 3.77 KB 1.00
Mockolate 1 510.0 ns 10.48 ns 9.81 ns 1.00 3.77 KB 1.00
Imposter 1 218.1 ns 11.12 ns 10.40 ns 0.43 1.19 KB 0.32
Moq 1 62,078.6 ns 1,601.02 ns 1,497.59 ns 121.77 20.26 KB 5.38
NSubstitute 1 5,318.8 ns 173.91 ns 162.68 ns 10.43 12.78 KB 3.39
FakeItEasy 1 5,613.6 ns 182.35 ns 170.57 ns 11.01 13.63 KB 3.62
baseline* 10 2,446.2 ns 14.59 ns 13.64 ns 1.54 4.82 KB 1.00
Mockolate 10 1,584.1 ns 28.24 ns 25.03 ns 1.00 4.82 KB 1.00
Imposter 10 519.5 ns 8.10 ns 7.18 ns 0.33 2.24 KB 0.47
Moq 10 64,765.9 ns 802.02 ns 710.97 ns 40.89 28.64 KB 5.94
NSubstitute 10 12,686.9 ns 188.32 ns 166.94 ns 8.01 25.58 KB 5.31
FakeItEasy 10 11,768.4 ns 470.52 ns 440.12 ns 7.43 32.97 KB 6.84

baseline* rows show the corresponding Mockolate benchmark from the most recent successful main branch build with results, for regression comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants