Repository navigation
Conversation
db9e915 to
bdd12a8
Compare
Pull-sync partners become provider-owned: browser ingest skips their sources and batch sync rejects them. The post-send dispatcher and the completeness marker are removed, and leftover marker cookies are expired. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The mirror cookie now holds only UIDs with a provider owner, so browser, batch and legacy pull-sync records are never mirrored, and its size budget counts the pending EID suffix for later UIDs. A pull-sync source with any live UID counts as held, as legacy pull sync did, so ownerless records no longer send the EC ID to the partner on every navigation. Config load warns when a pull-sync partner also sets a batch api_token, whose mappings batch sync rejects. Restore a test that an oversized pull-sync UID is dropped, and correct the guide on browser fallback ingest. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Batch sync rejects only sources owned by a configured identity provider. A source owned only by its partner's pull sync accepts batch mappings again, as it did before pull sync became a provider. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The mirror was re-set on every consented navigation with a provider UID, and any Set-Cookie makes the response private and uncacheable, so returning users lost all caching of publisher HTML. The request's mirror value is now kept and the cookie is set only when the new value differs. An unchanged mirror is not re-extended; once it lapses the next navigation sets it again. Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
…he cap Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
…rror exposure Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
bdd12a8 to
9f1e764
Compare
ChristianPavilonis
left a comment
There was a problem hiding this comment.
Summary
This PR moves pull sync into after-response identity collection and adds a bounded, JavaScript-readable mirror of provider-written bidstream UIDs. One startup-validation issue needs addressing; there is also an opportunity to avoid a redundant KV read on steady-state navigations.
The inline comment is prose-only: the proposed fix and regression coverage touch lines outside the diff, so there are no one-click suggestions.
Blocking
🔧 wrench
- Reject malformed pull tokens during startup — see inline at
crates/trusted-server-identity-providers/src/pull_sync.rs:59-60.
Cross-cutting / body-level findings
♻️ Avoid the follow-up KV read when all lookups were skipped
Introduced by crates/trusted-server-identity-providers/src/lib.rs:169-182; affected helper: crates/trusted-server-core/src/ec/after_response.rs:76-79.
When every pull-sync source already holds a UID, dispatch produces Skipped(NotDue) results. Those results are nonempty, so the follow-up helper rereads KV despite having nothing to persist. Moving existing pull-sync partners into this path adds an avoidable read on every eligible returning navigation.
Import LookupStatus and SkipReason alongside LookupResults, then replace the helper's empty-results guard with:
if lookups.iter().all(|lookup| {
lookup.entries.is_empty()
&& lookup.cleared.is_empty()
&& lookup.status != LookupStatus::Skipped(SkipReason::OptedOut)
}) {
return AfterResponseOutcome::Unchanged;
}This preserves writes that remember provider opt-outs.
Apply manually — the helper, imports, and regression test are outside the diff.
Verification
- A focused scratch reproduction confirmed that a newline-containing pull token passes runtime config validation but prevents construction of the coordinator for both an invalid and an otherwise valid partner.
- A counting-store scratch reproduction confirmed that all-
NotDueresults still cause one KV read in the follow-up helper. - Both reproductions passed. Scratch changes were removed and the worktree was restored to the reviewed head.
- The proposed fixes have not been compiled or run through the full local gate.
CI Status
- integration tests: PASS
- browser integration tests: PASS
- integration tests (Fastly EC lifecycle): PASS
- trace runtime and browser acceptance: PASS
- cargo fmt: PASS
- cargo test (axum native): PASS
- cargo test (cross-adapter parity): PASS
- format-docs: PASS
- worker-deadlines (spin): PASS
- worker-deadlines (cloudflare): PASS
- ts dev lint domains (changed lines): PASS
- cargo test: PASS
- format-typescript: PASS
- cargo test (ts CLI, native) (macos-latest): PASS
- HTML EOF runtime (Fastly): PASS
- cargo check/build/test (spin native + wasm32-wasip1): PASS
- CLAUDE.md symlink guard: PASS
- docs-proposal scripts: PASS
- CI Node pin guard: PASS
- cargo test (ts CLI, native) (ubuntu-latest): PASS
- cargo check (cloudflare native + wasm32-unknown-unknown): PASS
- prepare integration artifacts: PASS
- vitest: PASS
No required checks were reported by the required-check query.
| let mut value = http::HeaderValue::try_from(format!("Bearer {}", token.expose())) | ||
| .map_err(|_| Report::new(ProviderConfigError::Invalid))?; |
There was a problem hiding this comment.
🔧 Reject malformed pull tokens during startup
A pull token containing a newline passes runtime config validation but fails here. Because provider construction collects into one Result (src/lib.rs:169-182), that single invalid token prevents every after-response provider from running. The request-time caller merely logs the failure.
Reject invalid resolved tokens in crates/trusted-server-core/src/ec/registry.rs during startup, while retaining the disabled-partner bypass:
if require_nonempty_token
&& token_value.is_some_and(|value| {
value.trim().is_empty()
|| http::HeaderValue::try_from(format!("Bearer {value}")).is_err()
})
{
return Err(Report::new(TrustedServerError::Configuration {
message: "ts_pull_token must be nonempty and valid in an HTTP Authorization header"
.to_owned(),
}));
}Add regression coverage for invalid enabled tokens and disabled partners. Validate resolved secret values at runtime config load, not secret-store key references during deploy validation.
Apply manually — the fix and regression tests touch lines outside this PR's diff hunks.
A focused scratch reproduction confirmed the startup-acceptance/coordinator-failure sequence with two partners. The proposed fix itself has not been compiled.
Summary
__Host-ts-eids-mirrorcookie so publisher-side code can read them, on a best-effort basis.Stacked on #1487; review only the commits on top of it.
Changes
trusted-server-identity-providers/src/pull_sync.rs; removes coreec/pull_sync.rs/ec/pull_sync_marker.rsand the old post-send pull sync path in the Fastly adapter. Pull-sync providers are built withcollect = "after_response", so they only run for an EC whose consented row already existsec/mirror.rs;ec/finalize.rs,ec/cookies.rs,constants.rswrite the__Host-ts-eids-mirrorcookieapi_tokenkeeps batch syncSet-Cookie, so returning visitors' pages stay cacheablekind = "pull-sync"in[[ec.identity_providers]]is rejected; pull sync comes only from partner config with its token and rate limit__Host-ts-eids-mirrorjoins the identity cookies the template cache key refusesdocs/guide/edge-cookies.mdandconfiguration.mdBehaviour changes from
mainidentity_after_response_budget_ms, and only for an EC whose consented row already exists.pull_sync_concurrencyis a cap per navigation. It used to be a batch width with every eligible partner pulled; partners now rotate hourly so each gets turns under the cap.pull_sync_rate_limitis enforced per minute with a minimum of one on Fastly, so the default of 10 per hour behaves like 60.bidstream_enabledpartners they reach the script-readable mirror cookie. Previously they lived only in KV.Closes
Part of #1246 and #1245; the provider implementations that close them come in follow-up PRs.
Spec
Spec: reviewed separately; not included in this PR.
Test plan
cargo test-fastly && cargo test-fastly-reuse && cargo test-axum && cargo test-cloudflare && cargo test-spincargo fmt --all -- --checkscripts/lint-wasm-clock.sh./scripts/test-cli.sh, domain lint against the base branch, CI Node pin guard, docs-proposal scriptsfastly compute serveChecklist
unwrap()in production code — useexpect("should ...")logmacros (notprintln!)