Skip to content

chore: release 0.17.0 - #295

Merged
ethan-scitix merged 1 commit into
mainfrom
chore/release-0.17.0
Nov 14, 2026
Merged

ethan-scitix merged 1 commit into
mainfrom
chore/release-0.17.0

Conversation

@ethan-scitix

Copy link
Copy Markdown
Collaborator

Type

  • chore — CI, tooling, dependencies, config

Summary

  • Release 0.17.0. Unreleased moves to ## [0.17.0] - 2026-11-14, with a
    summary paragraph and the compare link.
  • The Dockerfile installs sieval-0.17.0-py3-none-any.whl.
  • This is a minor bump because feat(core)!: record exact error statuses and provider error codes #294 is breaking for anyone reading request
    summaries. An error status is now recorded as its exact code ("401",
    "402", "503", …), so new directories no longer contain 4xx or 5xx.
    error_code and error_codes are added. The on-disk format does not
    change: identity stays at schema 5 and RESULT_SCHEMA_VERSION stays at 1,
    and older directories resume as before.

Related Issues

Refs #293, #294

Test Plan

Automated

  • pre-commit run --files CHANGELOG.md Dockerfile passes.

  • python scripts/check_preflight.py --level deep: every check passes,
    including the reference-impl URL reachability check and the version checks.

  • ruff check . and ty check are clean.

  • pytest tests/unit tests/integration tests/acceptance -m "not benchmark":
    11392 passed, 11 skipped and 4 failed. All 4 failures come from this Mac's
    environment:

    • The three wikisql tests need a default locale. With LANG/LC_ALL
      set, all 44 pass.
    • The quotebench anchor test fails the same way on untouched
      origin/main, because it needs GNU tools.
    • CI on 76c675b is green.
  • Benchmark, run serially and without --cov, against v0.16.0 back to back
    under the same load, two rounds each. Every scenario passes, and the release
    matches v0.16.0 within noise:

    Scenario v0.16.0 SPS 0.17.0 SPS
    Single-turn eval 576.6 / 577.1 573.6 / 575.6
    Long output gen 160.3 / 161.5 160.0 / 161.1
    Multi-iteration (x3) 181.6 / 187.4 182.1 / 182.1
    High concurrency 1085.7 / 1092.1 1089.3 / 1087.3
    Framework overhead 1668.9 / 1646.2 1692.6 / 1646.4
    Resume (90%) 0.5 s 0.5 s

    An earlier run on a loaded machine failed four scenarios. That run was
    discarded as load, as the release skill allows.

Checklist

Required (all PRs)

  • PR title follows conventional format (type(scope): description)
  • No internal paths, credentials, or personal info in committed files
  • No new upper-layer dependencies added to core/

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ethan-scitix
ethan-scitix merged commit d8d3941 into main Nov 14, 2026
8 checks passed
@ethan-scitix
ethan-scitix deleted the chore/release-0.17.0 branch November 14, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant