Skip to content

feat(zest): position scanner for v0-8-market, finds real liquidatable accounts - #101

Open
mattglory wants to merge 8 commits into
mainfrom
zest-position-scanner
Open

mattglory wants to merge 8 commits into
mainfrom
zest-position-scanner

Conversation

@mattglory

@mattglory mattglory commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Enumeration: positions live in v0-market-vault (the persistent shared vault contract), not the market contract itself — that's why v0-4 → v0-8 never needed a position migration. get-nr() = total registered accounts (1,334), lookup(id) gives {account, mask, ...} for each.

Pricing architecture, confirmed by reading resolve-pyth/resolve-callcode directly rather than guessed: only 3 real Pyth Lazer feeds exist in the whole protocol (u1 sBTC, u7 USDC, u45 STX) — every other asset (the 7 v0-vault-* zToken/share tokens, stSTX, stBTC) derives its price from one of these via a callcode multiplier (ztoken index, ststx ratio, stbtc ratio), not an independent feed. USDH prices via a separate DIA oracle call — its "ident" field is a Clarity-serialized string ("USDh/USD"), not a Pyth hash.

Liquidation eligibility read directly from liquidate()'s own health check, not inferred: current-ltv = debt-usd * 10000 / collateral-usd, liquidatable when current-ltv >= LTV-LIQ-PARTIAL (the egroup's own threshold for that exact position mask).

Real bugs found and fixed while building and testing this against live data

  • cvToValue(cv, true)'s envelope behavior is genuinely inconsistent: a value bare at a function's own top-level return type (e.g. get-nr's plain uint, lookup's plain tuple) decodes with no {type,value} envelope; a value recovered from inside an unwrapped response/optional keeps one. Several decode sites assumed one pattern or the other without checking — normalized with a single uw() helper after confirming the actual shape of every call site empirically.
  • STSTX-RATIO-DECIMALS was assumed to be 1e8 by analogy with STBTC's own ratio scale — wrong, it's 1e6, caught because the resulting stSTX price came out ~84x too low ($0.0046 instead of ~$0.46). Re-read the real constant directly rather than keep guessing from the wrong one.
  • A full 1,334-account scan hit a bare ETIMEDOUT partway through — a transient network read timeout, not a rate limit, which threw directly out of fetch() and bypassed the JSON-parse-step retry entirely, killing the whole scan. Retry now wraps the fetch() call itself too. Also made the main loop resilient to a single account failing after retries are exhausted (logged and skipped, not fatal to the whole run).

Update: STX pricing fixed — no longer needs the Lazer entitlement at all

The original version of this PR was blocked on an external gap: this account's Pyth Lazer plan grants feeds 1 (sBTC) and 7 (USDC) but not 45 (STX) — confirmed by testing each feed individually ("Not entitled: feed 45"), requiring a loudly-warned STX_PRICE_USD8_FALLBACK manual escape hatch for every real run.

Checked whether a cheaper Lazer tier solves this instead of paying for Starter ($500/mo): it doesn't. Pyth Core (their free pull-oracle, Hermes) dropped Stacks support entirely on 2026-08-26 — Lazer is now the only Pyth product live on Stacks at all, and Lazer's own Free tier has no API access regardless.

But this scanner never submits anything on-chain — it only estimates risk off-chain — so it never actually needed Lazer's entitled feed in the first place. Zest's own v0-8-market already reads a second, independent oracle (DIA) for USDH; confirmed live that DIA also carries its own STX/USD feed (currently $0.3947, same USD x 1e8 scale as USDh/USD), free, no entitlement gate. Swapped the feed-45 Lazer call for a new fetchDiaStx() and removed STX_PRICE_USD8_FALLBACK entirely — no longer needed.

This does not help the real on-chain liquidate() submission (modes 3/4 of zest-v2-liquidation-receiver, PR #98) — Zest's own contract hardcodes verification against a genuine signed Lazer buffer for that call, so there's no substitute there regardless of cost. Only this scanner's off-chain estimation needed an alternative, and now has one for free.

Scope: this scans and flags candidates only — it does not liquidate anything. Exact liquidation sizing (how much debt to repay, the real bonus) uses Zest's own curve-based calc-liquidation-params, which this does not replicate; the receiver's own min-collateral-expected/min-profit checks are the real safety net for sizing precision, same as every other receiver built this session. Wiring a found candidate into zest-v2-liquidation-receiver's set-target/set-price-feed/flash-loan execution flow is a separate next step.

Test plan

  • Run against the complete live dataset, not a sample: all 1,334 accounts, 1,039 correctly skipped (no debt), 295 fully evaluated with real collateral/debt/price/LTV computation, 0 failures, 0 liquidatable right now — a plausible healthy state for Zest's current borrowers, not an untested claim
  • Verified the pricing math against hand-computed expectations at each step (sBTC/USDC/USDH/STX base prices, all 7 derived callcode prices) before trusting the enumeration loop
  • Verified resilience fixes by reproducing the original ETIMEDOUT failure mode and confirming the full rerun completed cleanly
  • node --check clean after the DIA pricing change; confirmed the exact get-value decode path against a live call (STX/USD → $0.394714, same field-access shape as the existing fetchDiaUsdh); confirmed no LAZER_TOKEN/feed-45/fallback references remain in the file
  • Not re-run end-to-end against the full 1,334-account dataset with a live LAZER_TOKEN in this environment (none available) — the sBTC/USDC Lazer calls are unchanged from the already-verified prior run, only the STX source changed and was independently confirmed correct

🤖 Generated with Claude Code

… accounts

Enumeration: positions live in v0-market-vault (the persistent shared
vault contract), not the market contract itself -- that's why v0-4 ->
v0-8 never needed a position migration. get-nr() = total registered
accounts (1,334), lookup(id) gives {account, mask, ...} for each.

Pricing architecture, confirmed by reading resolve-pyth/resolve-callcode
directly rather than guessed: only 3 real Pyth Lazer feeds exist in the
whole protocol (u1 sBTC, u7 USDC, u45 STX) -- every other asset (the 7
v0-vault-* zToken/share tokens, stSTX, stBTC) derives its price from one
of these via a callcode multiplier (ztoken index, ststx ratio, stbtc
ratio), not an independent feed. USDH prices via a separate DIA oracle
call -- its "ident" field is a Clarity-serialized string ("USDh/USD"),
not a Pyth hash.

Liquidation eligibility read directly from liquidate()'s own health
check, not inferred: current-ltv = debt-usd * 10000 / collateral-usd,
liquidatable when current-ltv >= LTV-LIQ-PARTIAL (the egroup's own
threshold for that exact position mask).

Real bugs found and fixed while building and testing this against live
data, not just syntax-checked:
- cvToValue(cv, true)'s envelope behavior is genuinely inconsistent: a
  value bare at a function's own top-level return type (e.g. get-nr's
  plain uint, lookup's plain tuple) decodes with no {type,value}
  envelope; a value recovered from inside an unwrapped response/optional
  keeps one. Several decode sites assumed one pattern or the other
  without checking -- normalized with a single uw() helper after
  confirming the actual shape of every call site empirically, not by
  further guessing.
- STSTX-RATIO-DECIMALS was assumed to be 1e8 by analogy with STBTC's own
  ratio scale -- wrong, it's 1e6, caught because the resulting stSTX
  price came out ~84x too low ($0.0046 instead of ~$0.46). Re-read the
  real constant directly rather than keep guessing from the wrong one.
- A full 1,334-account scan hit a bare ETIMEDOUT partway through -- a
  transient network read timeout, not a rate limit, which threw directly
  out of fetch() and bypassed the JSON-parse-step retry entirely,
  killing the whole scan. Retry now wraps the fetch() call itself too.
  Also made the main loop resilient to a single account failing after
  retries are exhausted (logged and skipped, not fatal to the whole run).

Run against the complete live dataset, not a sample: all 1,334 accounts,
1,039 correctly skipped (no debt), 295 fully evaluated with real
collateral/debt/price/LTV computation, 0 failures, 0 liquidatable right
now -- a plausible healthy state, not an untested claim.

One real, external blocker, not a code issue: this account's Pyth Lazer
plan grants feeds 1 (sBTC) and 7 (USDC) but not 45 (STX) -- confirmed by
testing each feed individually ("Not entitled: feed 45"). STX is the
base price for wSTX debt and everything that derives from it (stSTX,
every zstx/zststx/zststxbtc vault share), so a fully live scan needs
that grant before this can be trusted for a real decision.
STX_PRICE_USD8_FALLBACK is an explicit, loudly-warned escape hatch for
testing everything else in the meantime -- every real run above used it
and says so in its own output.

Scope: this scans and flags candidates only -- it does not liquidate
anything. Exact liquidation sizing (how much debt to repay, the real
bonus) uses Zest's own curve-based calc-liquidation-params, which this
does not replicate; the receiver's own min-collateral-expected/
min-profit checks are the real safety net for sizing precision, same as
every other receiver built this session. Wiring a found candidate into
zest-v2-liquidation-receiver's set-target/set-price-feed/flash-loan
execution flow is a separate next step.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
web Ready Ready Preview Oct 12, 2026 5:40pm UTC

Request Review

The scanner's STX price was blocked on Lazer feed 45, which this
account's Demo plan doesn't grant (confirmed live: "Not entitled: feed
45"), forcing a manual STX_PRICE_USD8_FALLBACK escape hatch for any real
use. Checked whether a cheaper Lazer tier exists instead of paying for
Starter ($500/mo): it doesn't -- Pyth Core (their free pull-oracle,
Hermes) dropped Stacks support entirely on 2026-08-26, so Lazer is now
the only Pyth product live on Stacks at all, and Lazer's own Free tier
has no API access regardless.

But this scanner never submits anything on-chain -- it only estimates
risk off-chain -- so it never needed Lazer's entitled STX feed in the
first place. Zest's own v0-8-market already reads a second independent
oracle (DIA) for USDH; confirmed live that DIA also carries an
independent STX/USD feed (currently $0.3947, same USD x 1e8 scale as
USDh/USD), free and with no entitlement gate. Swapped fetchLazerFeeds([45])
for a new fetchDiaStx(), removed the STX_PRICE_USD8_FALLBACK hatch
entirely (no longer needed), and updated the header's pricing
architecture section to match.

This does NOT help the real on-chain liquidate() submission (modes 3/4
of zest-v2-liquidation-receiver) -- Zest's own contract hardcodes
verification against a genuine signed Lazer buffer for that call, so
there's no substitute there regardless of cost. Only this scanner's
off-chain estimation needed an alternative, and now has one for free.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mattglory added a commit that referenced this pull request Oct 11, 2026
Pulls in PR #101's fix (price STX via DIA instead of Pyth Lazer's
gated feed 45) so the keeper bot's resolveAllPrices() call gets a real
STX price without needing the Lazer Starter plan upgrade.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The liquidation keeper bot (PR #102) needs this to decode Velar pool
reads the same way the scanner already decodes Zest's own contract
calls -- was a plain top-level function, not exported.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Primary fix: readOnly()'s fetch() call had no timeout at all. Caught
live -- a long-running watch loop (PR #102's keeper) hung completely for
37+ minutes on a single stuck call: 0% CPU, no open socket, zero log
output, stuck on an await that was never going to resolve or reject on
its own. Wrapped the call in an AbortController, 20s. Verified in
isolation against a deliberately unroutable address (the exact hang
shape, not a refused connection) that this actually throws a catchable
AbortError instead of hanging forever -- the existing retry logic
already handles that case correctly, it just never got the chance to
before.

Also pulls in the rest of this file's `export` surface from the keeper
branch (BPS, DEBT_OFFSET, ASSET_TABLE, ASSET_BY_ID, readOnly,
resolveAllPrices, getNr, lookup, maskHasDebt, maskAssetIds, getCollateral,
getDebtScaled, getCachedIndex, getEgroup, evaluateAccount) -- only `uw`
had been synced back here before. No behavior change, keeps this branch
as the single source of truth for the file both #101 and #102 depend on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mattglory

Copy link
Copy Markdown
Owner Author

Found and fixed a real reliability bug while running #102's keeper live: `readOnly()`'s `fetch()` call had no timeout at all. A long-running watch loop hung completely for 37+ minutes on a single stuck call -- 0% CPU, no open socket, zero log output, stuck on an `await` that was never going to resolve or reject on its own.

Wrapped the call in an `AbortController`, 20s. Verified in isolation against a deliberately unroutable address (the exact hang shape -- a connection that never responds, not one that's refused) that this actually throws a catchable `AbortError` instead of hanging forever. The existing retry logic already handled that case correctly; it just never got the chance to fire before.

Also synced this file's full `export` surface to match the keeper branch (only `uw` had made it back here before) -- no behavior change, just keeps this branch the single source of truth.

Caught live: a Lazer feed 1/7 timeout killed an entire WATCH-mode keeper
process outright (PR #102), seconds after restart. fetchLazerPrices()
called process.exit(1) on that failure -- appropriate for this script's
own one-shot main() (nothing to retry, exit loudly), but process.exit()
isn't a catchable error, so it bypassed the keeper's loop try/catch
entirely instead of letting it log and retry like any other transient
failure.

Now throws instead. This script's own one-shot main() still exits the
same way via its existing top-level .catch(...) -> process.exit(1)
wrapper -- unchanged behavior there. The missing-LAZER_TOKEN case stays a
hard process.exit(1): a permanent config error, not transient, so
hard-exiting immediately is correct in both contexts.

Also fixed a stale URL in the same error message (pythdata.app ->
app.pyth.com).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mattglory

Copy link
Copy Markdown
Owner Author

Second reliability fix from running #102's keeper live: a transient Lazer feed timeout was killing the entire WATCH-mode process outright, not just failing one scan cycle. `fetchLazerPrices()` called `process.exit(1)` on that error -- correct for this script's own one-shot run, but `process.exit()` isn't catchable, so it bypassed the keeper's loop `try/catch` entirely.

Now throws instead; the one-shot `main()` still exits the same way via its existing top-level wrapper. The missing-token case stays a hard exit (permanent config error, not transient).

Caught live: feeds 1/7 (sBTC/USDC) -- previously confirmed working --
timed out repeatedly on a fresh restart, with nothing but a generic
"timed out waiting for Lazer feeds" message each time. fetchLazerFeeds
muted the SDK's own logger entirely (including error/warn) and the
WebSocket pool's onError callback, so whatever the REAL cause is (auth
rejection, DNS, TLS, a protocol-level issue) had no way to surface -- a
connection that fails outright and one that just never responds look
identical from the caller's side.

error/warn now print (debug/info stay muted, too noisy), onError prints
the real error instead of discarding it, and PythLazerClient.create()
itself gets an explicit .catch() in addition to the existing
unhandledRejection safety net.

Diagnostic-only change -- doesn't fix whatever the underlying issue
actually is, just stops hiding it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mattglory

Copy link
Copy Markdown
Owner Author

@unixwhisperer #98 is merged. This one's next in the stack -- scanner only, no execution: finds liquidatable Zest accounts off-chain, doesn't touch the receiver. Same discipline as #98: pricing math/decode shapes verified against live calls, not assumed (full writeup in the PR body, including the STSTX-RATIO-DECIMALS and ETIMEDOUT-retry bugs it caught along the way).

One thing worth your eyes specifically: the STX-pricing switch from Lazer feed 45 to DIA -- this scanner never submits on-chain, so it doesn't need Zest's entitled feed, but want your confirmation that's a sound boundary (scanner estimates off DIA, the receiver in #98 still verifies against a real signed Lazer buffer on-chain, nothing crosses that line).

Ready for review whenever you have a minute.

@unixwhisperer unixwhisperer left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. The pricing derivation and LTV formula match v0-8-market, and the timeout, no-process.exit and diagnostics fixes are the right calls.

One fix before merge: getCollateral/getDebtScaled (L397-408) turn any error into 0n, including readOnly running out of network retries, not just the documented "no entry" case. A collateral read that fails this way gives collateralUsd = 0, so currentLtv = BPS, and the account is flagged as liquidatable (L461-465). The keeper would then arm a healthy position and burn gas until Zest rejects it with ERR-HEALTHY. Please return 0n only for !res.okay and rethrow network errors. lookup (L372-378) should count its failures the same way instead of dropping the account silently.

Minor: STX is priced from DIA here, but Zest checks against Lazer feed 45 on-chain, and near the threshold the two can disagree. Before anything acts on a candidate close to the threshold, I'd require a small margin (say 50 bps) over LTV-LIQ-PARTIAL.

getCollateral/getDebtScaled caught every readOnly() failure -- including
network errors after retries are exhausted, not just the genuine "no entry
for this asset" case -- and returned 0n either way. A failed read on a
healthy account's collateral leg pushed its LTV to 100%, flagging it as
liquidatable. Added NoEntryError to mark only the real no-entry case
(res.okay === false); anything else now propagates.

lookup() had the same shape but no benign case at all -- every id in
[0, get-nr()) is guaranteed an entry, so a lookup failure is always a
network/parse issue. It no longer swallows it; main()'s loop now wraps
lookup() and evaluateAccount() in one try/catch so a lookup failure is
counted in `failed` and logged, instead of silently `continue`-ing with
no record.

Verified live: a 50-account run still completes clean (0 failed, 0
false positives) after the change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
mattglory added a commit that referenced this pull request Oct 12, 2026
…hardening

Four fixes from review, all before anything runs with EXECUTE=true:

1. Fetch the Lazer feed before broadcasting set-target, not after. The old
   order armed the receiver with a target and paid set-target's fee even
   when the feed fetch then failed or timed out -- every watch cycle, for
   nothing. Mode 4 (needs feed 45, which the active Demo plan doesn't
   grant) is now gated behind FEED_45_AVAILABLE and skipped outright
   rather than attempted and left half-armed.

2. Nonce handling: a second candidate in the same scan pass was re-reading
   the *confirmed* nonce, colliding with the first candidate's still-
   pending transactions and aborting the rest of the pass. Signer, the
   owner check, and the starting nonce (now possible_next_nonce, which
   accounts for pending txs) are derived once per pass and threaded
   through every candidate instead.

3. Minimum profit: the keeper never called set-min-profit-stx/-sbtc, so
   the floor stayed at the contract default (u1 -- any nonzero profit).
   Each attempt costs ~0.4 STX in fees (50k + 50k + 300k); a smaller
   liquidation would execute at a net loss. Set once per pass, both
   floors at 2x that cost, the sBTC one converted via this pass's own
   live prices rather than a guessed sat count.

4. Hardening: an explicit zero-send post-condition on flash-loan (signer
   should never move STX/sBTC out of their own wallet for this call, only
   the receiver's internal transfers); renamed DEPLOYER_MNEMONIC to
   KEEPER_MNEMONIC and added a live get-owner() check before broadcasting
   anything, matching #98's ownership-handoff plan; ported #101's Lazer
   diagnostics + explicit .catch fix into fetchLazerEvmBuffer, which was
   still muting every error.

Also merged in #101's own fix (zest-position-scanner, 9663da6) since this
file imports evaluateAccount/lookup directly from it -- the keeper would
otherwise still have the same false-liquidatable-on-network-blip bug.
scan()'s own duplicate account loop gets the matching one-try/catch fix.

Verified: node --check on both files; a 50-account dry run against live
data completes clean (0 failed, matching the scanner's own verification).
Not run end-to-end with EXECUTE=true -- the receiver isn't deployed yet
(PR #98 merged, not yet broadcast) and there's nothing live to act on.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 9663da6e Deployed Oct 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants