[gargsaumya]

Work Item / Issue Reference

AB#49646

---

Summary

Replace the legacy Rust-runtime NuGet transport with exact-version Python wheels.

• Local and official/release builds default to the committed released runtime from public PyPI.
• GitHub PR validation, which executes in Azure DevOps, resolves the latest published scheduled nightly from the Azure Python feed once per run. Main-branch runs of that validation pipeline follow the same development policy.
• All ten validation build/test jobs use that one exact version; the rs-runtime-selection artifact records the selection and nightly wheel URLs/hashes.
• Feature testing can still select an exact published version through MSSQL_PYTHON_RS_VERSION or rsVersionOverride, taking precedence over automatic nightly selection.
• Use the same selected version in dependency metadata, host/Docker installation, and pre-test version verification.
• Preserve cross-architecture Windows validation and Conda artifact provenance.
• Block official releases of prerelease runtimes or override-built artifacts.
• Install the exact metadata-pinned ODBC dependency from the same selected build in both stress jobs.

Review follow-up also isolates source-pin packaging tests from inherited overrides,
declares their wheel build dependency, and selects toolcache Python 3.12 for coverage
instead of relying on the hosted OS Python's global pip configuration. Intentional
negative-test scanner literals have rule-specific inline suppressions.

The second review follow-up hides both binding and ODBC checkout packages before
stress imports and verifies that ODBC resolves outside the checkout before loading
the binding. Pipeline regressions require their runtime/Conda helper sources, skip
incomplete wheel fixtures, and fail rather than skip when PyYAML is missing from a
complete checkout. PyYAML is declared in the applicable test/build dependency
inputs and all four affected locks; existing locked versions are unchanged.

Automatic nightly selection recognizes scheduled X.Y.Z.devYYYYMMDD releases,
not per-merge versions with an appended build ID. It requires the newest nightly
to have all nine supported cp310-abi3 wheels with valid HTTPS URLs and SHA-256
fragments, and fails on missing, yanked, duplicate or incomplete publications
instead of silently using an older nightly or stable release. Official/release,
daily-validation and stress entry points retain their stable defaults and exact
override support. A feed publication is not an assertion that the producer's
overall build passed, nor proof of a particular Rust commit.

Validation

• First review follow-up: 1,471 affected packaging/runtime/pipeline/audit tests passed; 28 platform-dependent skips.
• First review follow-up: 1,328 affected tests also passed with an inherited dev-runtime override; 23 skips.
• Second review follow-up: 85 pipeline/runtime tests passed both without and with a dev override.
• Automatic nightly follow-up: 172 nightly-selection, pipeline, runtime and wheel-download tests passed both without and with an inherited exact dev override.
• Live selector CLI resolved 0.3.0.dev20261027 from the Azure Python feed, validated all nine wheel links/hash fragments, emitted the ADO output variable and wrote the selection artifacts.
• Existing cross-architecture installer downloaded and metadata-validated the selected nightly's 10.1 MB ARM64 wheel without installing it into the x64 host.
• Adjacent Conda/packaging/download regressions: 621 passed, one platform skip; focused profiler pipeline/daily/runtime checks: 32 passed, five platform skips.
• Adjacent release/binary-audit/packaging fixtures: 697 passed, 22 platform skips.
• The new import-provenance and incomplete-fixture regressions failed before the fixes; actual PowerShell and Git Bash isolation steps passed with synthetic installed packages.
• Four hash-required Python 3.10 platform dependency dry-runs passed; all 73 PyYAML SHA-256 hashes match PyPI, and repeated pinned-uv regeneration was stable.
• Conda release regression fails with the override guard disabled and passes with it enabled.
• Local DevSkim reports none of the three intentional test findings with targeted suppressions; disabling suppressions reproduces all three.
• Pinned Black checks and git diff --check passed.
• New and changed Python selection/pipeline tests parse with Python 3.10 syntax.
• Real dev wheel installation/native import verified through PowerShell and Git Bash.
• ARM64 cross-target download and all nine wheel hashes/metadata verified.
• Native SQL feature tests and live stress jobs were not executed locally.
• The broader profiler module cannot run fully in this checkout: ten tests require the unbuilt cp314 native binding. The changed pipeline's profiler contract passes.
• Latest Azure PR validation must pass before this draft is marked ready.

The previous Azure run's Debian/ARM64 failures were missing wheel in the metadata
tests; coverage failed at runtime installation with PEP 668 on OS-managed Python.
Both causes are addressed in this follow-up; remote results still need confirmation.

Public-release download smoke testing was blocked by local TLS connectivity to files.pythonhosted.org; TLS verification remains enabled.
The second follow-up's dependency locks were regenerated through the Microsoft
package proxy because of the same public package-host TLS limitation. Canonical
index and native-platform lock confirmation remains with CI.

Rollout

Coordinate with the companion mssql-rs release-worker PR before the next Rust release: microsoft/mssql-rs#772.

This is a draft pending CI and review.

[github-actions]

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task	Before	After	Paired change	Result
Connection opening	10.248 ms	10.166 ms	-0.3%	no signal
SELECT queries	1.069 ms	1.088 ms	+1.7%	no signal
Row insertion	34.402 ms	34.747 ms	+1.0%	no signal
Executemany inserts	160.714 ms	159.927 ms	-0.0%	no signal
Fetch-all queries	119.553 ms	120.103 ms	+1.4%	no signal
Row-by-row fetching	14.386 ms	14.398 ms	-0.5%	no signal
Batched row fetching	126.184 ms	116.181 ms	-8.4%	no signal
Transaction commit and rollback	114.353 ms	114.579 ms	+0.6%	no signal
Arrow row fetching	92.410 ms	93.301 ms	+1.0%	no signal
100,000-row insertion	448.251 ms	486.667 ms	+3.2%	no signal
Row fetching in batches of 100	121.197 ms	121.863 ms	+0.3%	no signal
Row fetching in batches of 10,000	134.600 ms	134.338 ms	+0.7%	no signal
Repeated positional queries	34.004 ms	33.617 ms	-1.2%	no signal
Repeated named-parameter queries	36.397 ms	36.092 ms	-0.6%	no signal
Legacy 100,000-row insertion	352.572 ms	358.311 ms	-1.9%	no signal
Insertion with explicit input sizes	483.880 ms	481.668 ms	-0.2%	no signal
Joined aggregation queries	176.472 ms	176.426 ms	-0.9%	no signal
Large joined-result fetching	183.710 ms	175.996 ms	-5.1%	no signal
1.2-million-row fetching	3427.000 ms	3460.192 ms	-0.1%	no signal
Common table expression queries	5.421 ms	5.317 ms	-2.5%	no signal
256 KiB VARCHAR(MAX) / fetchall()	1.260 ms	1.237 ms	-3.8%	no signal
10,000 scalar values / fetchval() (debug disabled)	106.722 ms	107.573 ms	-0.5%	no signal

Unix / SQL Server 2025

Database task	Before	After	Paired change	Result
Connection opening	97.402 ms	98.592 ms	+1.2%	no signal
SELECT queries	1.192 ms	1.215 ms	+1.9%	no signal
Row insertion	33.322 ms	33.722 ms	+1.6%	no signal
Executemany inserts	153.866 ms	154.981 ms	-0.8%	no signal
Fetch-all queries	125.672 ms	127.267 ms	+2.5%	no signal
Row-by-row fetching	14.474 ms	14.024 ms	-3.0%	no signal
Batched row fetching	120.079 ms	121.117 ms	+0.8%	no signal
Transaction commit and rollback	109.616 ms	109.207 ms	-1.1%	no signal
Arrow row fetching	96.899 ms	97.604 ms	-1.2%	no signal
100,000-row insertion	419.015 ms	477.827 ms	+9.8%	no signal
Row fetching in batches of 100	120.195 ms	125.873 ms	+5.0%	no signal
Row fetching in batches of 10,000	137.784 ms	139.334 ms	+1.4%	no signal
Repeated positional queries	32.749 ms	32.751 ms	-0.0%	no signal
Repeated named-parameter queries	35.724 ms	34.718 ms	-0.2%	no signal
Legacy 100,000-row insertion	340.153 ms	341.797 ms	-0.9%	no signal
Insertion with explicit input sizes	493.049 ms	505.406 ms	+2.7%	no signal
Joined aggregation queries	160.456 ms	160.144 ms	-0.3%	no signal
Large joined-result fetching	194.325 ms	192.658 ms	-0.9%	no signal
1.2-million-row fetching	3875.838 ms	3937.272 ms	+0.3%	no signal
Common table expression queries	6.208 ms	5.494 ms	+3.3%	no signal
256 KiB VARCHAR(MAX) / fetchall()	1.719 ms	1.472 ms	-14.6%	no signal
10,000 scalar values / fetchval() (debug disabled)	108.728 ms	106.017 ms	-2.2%	no signal

Build and measurement details

ADO build 186152

PR head: 0efc34c29b1321324f90c414b95c916374f2a0ad
Base: 185bc651fd2626aa4058de1be7658e52aad4c413
Measured merge: 46cb8cc6ad0f5dbba1ddbb6f7afbd580a03c75e4

• Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4305.5; 5 paired comparisons and 1 warmup.
• Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5035.5; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

[Copilot]

🟡 Changes recommended

Release eligibility enforcement and override/stress validation still have unresolved correctness issues.

3 open issues

• Reject version overrides in Conda release validation · New
• Install pinned ODBC dependency in stress jobs · New
• Clear version override in source-pin packaging fixture · New

What changed in this PR

This PR replaces NuGet transport for the driver's separate Rust runtime with exact-version Python wheels, using a committed PyPI release by default and explicit overrides for feature testing.

Changes:

• Centralizes runtime selection across package metadata, installers, and pre-test verification.
• Propagates overrides through build/test pipelines and records artifact provenance for release validation.
• Updates Conda handling, contributor documentation, and packaging/pipeline contract tests.

File	Description
tests/​test_040_rs_runtime_selection.py	Tests selection, installation, metadata, and verification.
tests/​test_039_rs_runtime_pipelines.py	Adds pipeline and release-gate contracts.
tests/​test_038_mssql_odbc_daily_validation.py	Updates pin and container-override assertions.
tests/​test_037_rs_wheel_download.py	Tests index downloads, hashes, and metadata checks.
tests/​test_036_conda_provenance.py	Updates runtime receipt and source-provenance tests.
tests/​test_034_conda_verify_cwd.py	Tests producer-version and wheel-hash validation.
tests/​test_030_pe_machine_assert.py	Adds the resolver to the packaging fixture.
setup.py	Uses the shared resolver for runtime dependencies.
OneBranchPipelines/​stress-test-pipeline.yml	Selects build artifacts and verifies runtime versions.
OneBranchPipelines/​steps/​conda-release-step.yml	Removes obsolete transport-version inputs.
OneBranchPipelines/​steps/​conda-publish-step.yml	Removes obsolete transport-version forwarding.
OneBranchPipelines/​stages/​build-windows-single-stage.yml	Adds Official-build guards and runtime verification.
OneBranchPipelines/​stages/​build-macos-single-stage.yml	Adds Official-build guards and runtime verification.
OneBranchPipelines/​stages/​build-linux-single-stage.yml	Forwards overrides and updates installed-wheel validation.
OneBranchPipelines/​scripts/​validate-rs-wheel-dependency.ps1	Cross-checks dependency versions against receipts.
OneBranchPipelines/​scripts/​validate-release-versions.ps1	Validates release eligibility and public PyPI availability.
OneBranchPipelines/​official-release-pipeline.yml	Passes artifact receipts to release validators.
OneBranchPipelines/​jobs/​consolidate-artifacts-job.yml	Stages effective runtime versions, wheels, and receipts.
OneBranchPipelines/​dummy-release-pipeline.yml	Passes receipts through test release validation.
OneBranchPipelines/​conda-release-pipeline.yml	Removes legacy transport provenance variables.
OneBranchPipelines/​build-release-package-pipeline.yml	Exposes the explicit runtime override parameter.
MANIFEST.in	Includes the resolver in source distributions.
eng/​versions/​mssql-python-rs-nuget.version	Removes the NuGet transport pin.
eng/​scripts/​mssql_python_rs.py	Introduces shared selection and installed-runtime verification.
eng/​scripts/​install-mssql-py-core.sh	Delegates installation to the Python installer.
eng/​scripts/​install-mssql-py-core.ps1	Delegates installation and cross-architecture validation.
eng/​scripts/​install_mssql_python_rs.py	Downloads and installs the selected runtime wheel.
eng/​scripts/​download_mssql_python_rs_wheels.py	Replaces NuGet extraction with verified index downloads.
eng/​pipelines/​steps/​install-mssql-py-core.yml	Forwards runtime selection into container installation.
eng/​pipelines/​pr-validation-pipeline.yml	Propagates overrides and verifies runtimes before tests.
eng/​pipelines/​mssql-odbc-daily-validation-pipeline.yml	Adds override propagation and runtime preflight.
eng/​conda_tools/​release.py	Uses the updated runtime receipt validator.
eng/​conda_tools/​provenance.py	Removes NuGet transport-pin provenance.
eng/​conda_tools/​inputs.py	Validates distribution pins and index-based receipts.
eng/​conda_tools/​build.py	Reuses receipt validation for staged runtime inputs.
CONTRIBUTING.md	Documents runtime selection and override restrictions.
conda/​README.md	Documents index-based runtime provenance.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Stress jobs still allow checkout ODBC packages to shadow installed artifacts, and the new regression tests have dependency and fixture gaps.

3 open issues

• Hide checkout ODBC package before stress-job imports · New
• Skip incomplete wheel fixtures missing runtime helpers · New
• Declare PyYAML and require regression checks in CI · New

3 resolved since last review

• Reject version overrides in Conda release validation
• Clear version override in source-pin packaging fixture
• Install pinned ODBC dependency in stress jobs

_{Review effort: Balanced}

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 85%

📈 Total Lines Covered: 9444 out of 11093 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%
mssql_python.cursor.py: 92.5%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

🔵 Needs a closer look

Cross-platform packaging and release gates need human review and confirmed Azure validation, with a downloader issue still unresolved.

1 open issue

• Declare PyYAML and require regression checks in CI

1 previously missed issue

In code that hasn't changed since last review

Support dot-separated platform tags in wheel selection

eng/​scripts/​download_mssql_python_rs_wheels.py:81

The platform pattern silently skips valid wheels with dot-separated platform tags, such as manylinux_2_34_x86_64.manylinux_2_35_x86_64. If the index also contains single-tag wheels, staging succeeds while omitting these Linux wheels, although the Conda selector accepts them (eng/conda_tools/contracts.py:207). Allow dot-separated platform tags and add a download regression case with both single-tag and multi-tag wheels.

2 resolved since last review

• Skip incomplete wheel fixtures missing runtime helpers
• Hide checkout ODBC package before stress-job imports

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The cross-platform packaging and release-gate migration needs human sign-off, with latest Azure validation and live native/stress execution still unconfirmed.

1 open issue

• Declare PyYAML and require regression checks in CI

_{Review effort: Balanced}