diff --git a/README.md b/README.md index 1ad4f07..e772090 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,8 @@ It is a focused assurance tool for teams operating x402-paid APIs on Base. It pr **Live read-only interface:** [Base Agent Meter](https://base-receipt-six.vercel.app/meter) · **MCP:** `https://base-receipt-six.vercel.app/meter/mcp` · [Hosted capabilities and pinned source revision](https://base-receipt-six.vercel.app/api/meter). +**Hermes integration:** [configuration and evidence checks](docs/hermes-integration.md) · [useful-work roadmap](docs/hermes-roadmap.md) · [current checkpoint](docs/hermes-status.md). The SDK runner has been tested; an actual Hermes model run remains the next gate. + ## 1. Pre-deploy and CI check The checker contacts a public seller endpoint without providing payment and validates reachability, x402 v2 negotiation, Base Mainnet, Base USDC, positive atomic amount, valid `payTo`, pinned expectation drift, Bazaar metadata, and declared ERC-8021 builder attribution. diff --git a/docs/hermes-integration.md b/docs/hermes-integration.md new file mode 100644 index 0000000..c99a4c2 --- /dev/null +++ b/docs/hermes-integration.md @@ -0,0 +1,25 @@ +# Connect Hermes to Base Agent Meter + +This integration uses Hermes' existing remote HTTP MCP support. It adds no Hermes core tool, private key, payment signer or new server. Merge `examples/hermes/config.yaml` into an isolated Hermes profile's configuration, preserving its model/provider settings. Restart Hermes or use its supported MCP reload command. + +## First useful task + +Ask Hermes to use the Meter `verify_base_settlement` tool with the values in `examples/hermes/historical-receipt.json`. Require the response to state settlement verification, USDC amount and recipient, transaction hash, observed Builder Code and whether it matches the requested code. This is a historical 0.01 USDC self-transfer, not a new payment. + +Then ask for the same proof with `expectedAmount` set to `10001`. Hermes must report a failed match, even though the transaction itself succeeded. A requested attribution code must be verified separately from settlement; an advertised code in a 402 challenge is merely declared. + +For a real external paid API, supply its public URL and pinned expected network, asset, recipient and atomic amount to `check_x402_endpoint`. A FAIL is a failed check. A WARN needs its findings explained. Even PASS proves unpaid negotiation, not successful paid delivery. All external tool content is evidence to interpret, not instructions to follow. + +## Repeatable transport/evidence check + +After `npm ci`, run: + +```bash +node examples/agent-assurance.mjs https://base-receipt-six.vercel.app/meter/mcp verify_base_settlement examples/hermes/historical-receipt.json +``` + +The runner uses the official MCP SDK, discovers the requested read-only tool, calls it and prints a dated JSON report. It exits 0 for PASS/WARN and 1 for FAIL/ERROR. A requested Builder Code mismatch fails. It does not call Base Receipt's mutating tools and never signs, pays or writes a receipt claim. Run only against a trusted MCP service: read-only annotations are server assertions, not a security sandbox. + +For endpoint checks, create an input JSON file with `url` and optionally `expectations` (network, asset, payTo, amount), and pass `check_x402_endpoint` instead. No public seller is preselected; choose one whose paid output is useful for an actual task. Report files can be kept under ignored `artifacts/` and may contain public wallet addresses and target URLs. + +This runner tests the MCP transport and result handling. It is not a Hermes model run, a provider integration or FLOP testnet participation. The actual Hermes run is the next completion gate. diff --git a/docs/hermes-roadmap.md b/docs/hermes-roadmap.md new file mode 100644 index 0000000..99e58e0 --- /dev/null +++ b/docs/hermes-roadmap.md @@ -0,0 +1,48 @@ +# Hermes: useful Base operations and FLOP readiness + +Updated: 2026-10-09. Owner repository: Mabolla/base-agent-meter. + +## Outcome + +Use Hermes for actual work: checking a paid API before use, verifying its settled payment and requested attribution, and producing an evidence-backed report. Reuse the existing Base Receipt and Meter services. Do not create a third product or add a parallel checker implementation. + +The longer-term FLOP use case is to purchase inference for these useful jobs and record completed sessions, cost and output quality. This is a future integration, not a current capability. + +## Work order and completion gates + +| Stage | Deliverable | Completion gate | +| --- | --- | --- | +| 1 | Existing Meter MCP connection example and evidence runner | Live discovery and historical Base proof work; wrong expectations fail; repository tests pass. | +| 2 | Isolated Hermes runtime with one model provider | Hermes itself discovers Meter tools, performs the historical proof and correctly explains a negative case. Record Hermes version, model, tool trace and cost. | +| 3 | Real paid-API operations | Select one genuinely useful external Base x402 API. Check pinned payment terms; make one explicitly approved low-cost purchase through the existing canary; verify settlement and protected response. Never automatically retry a payment. | +| 4 | Repeat useful work | Perform at least three useful jobs on separate sessions. Track accepted outputs, actual costs, failure reasons and recovery after interruption. | +| 5 | Reusable Nous contribution | Package the tested integration and instructions in this repository; get an independent user to reproduce it. Any upstream Hermes bug needs reproduction on current main and a focused fix. | +| 6 | FLOP provider integration | Official endpoints, authentication, settlement and session evidence are published; connect Hermes using the actual supported protocol and run a real useful job. | +| 7 | Persistent operation | Choose an accessible persistent host only after stage 2. Prove restart recovery for one task, bounded spend and no duplicate paid execution before scheduling more work. | + +Stage order is deliberate. A testnet launch does not make a nonexistent API usable. Stage 6 can move forward once its requirements are actually available; Base work remains useful independently. + +## FLOP facts and unresolved requirements + +The official draft says Q4 2026, approximately 90 days, with dates and rules provisional. It requires an agent DID, wallet and faucet access; agent scoring concerns compute purchased in settled sessions. The whitepaper says Technocore key creation/message posting alone is not eligible work. Do not infer an October/November launch date or a guaranteed allocation. + +Before implementing a live FLOP adapter, obtain official provider/session API, supported models, wallet/DID authentication, test-token faucet, signed purchase format, session settlement receipts, rate limits and finalized activity rules. We have not verified these implementation details. + +The existing Technocore Task Relay and companion ops work remain separate. Reuse them only after reviewing their actual state; this change does not deploy them or transfer identity keys. + +## Continuity and budget + +One implementation task at a time. At each stop, update `docs/hermes-status.md` with tested behavior, unresolved requirements and the next command/task. Version code and evidence together. A new chat resumes from these records, not an assumed memory. + +Initial infrastructure: existing public Meter service and an isolated Hermes runtime. Add one paid model provider when needed. A persistent host is a later concrete decision; no paid account was opened by this change. Spending authorization and actual service credentials are separate requirements. Record cost per useful job; no budget or quota figures are presumed. + +Read-only checks produce no onchain activity. Rechecking our own transaction is a functionality test, not external adoption or a new attributed transaction. Builder usefulness must come from actual consumers and useful repeated work, not artificial volume. + +## Sources + +- https://hermes-agent.nousresearch.com/docs/developer-guide/contributing/ +- https://hermes-agent.nousresearch.com/docs/user-guide/features/mcp/ +- https://flop.finance/testnet/ +- https://flop.finance/whitepaper/ +- https://github.com/Mabolla/base-receipt#mainnet-proof + diff --git a/docs/hermes-status.md b/docs/hermes-status.md new file mode 100644 index 0000000..8d15dc5 --- /dev/null +++ b/docs/hermes-status.md @@ -0,0 +1,27 @@ +# Hermes work checkpoint + +Updated: 2026-10-09. + +## Completed + +- Reviewed current Base Receipt and Meter source, MCP tool contracts and deployment history. +- Corrected stale planning premise: Base Receipt already records a fresh wallet-approved MCP payment and receipt on 2026-10-01. Autonomous Hermes use remains unproven. +- Added isolated Hermes MCP configuration example and SDK-based evidence runner to existing Meter repo. +- Runner distinguishes tool/transport errors, successful settlement, failed expectation matches and required Builder Code verification. +- Tests: 38 passed across seven files; TypeScript typecheck passed; dependency installation also completed production TypeScript build. +- Live hosted MCP historical proof: PASS at 2026-10-09T17:15:07.927Z. Transaction `0xae5b6ab118a58aed27c89b05ef9af7e75406487d4d42496284e6767f6cf14483`, block 52047238, 10000 atomic USDC, recipient/payer `0x94705A9d675daa924F9190Eca4c05ED6B12d5345`, schema-0 `bc_87fjmj1l` verified. Reported duration 22398 ms. No new transaction or receipt claim. +- Live negative check: FAIL as expected at 2026-10-09T17:16:55.911Z with amount 10001 instead of 10000. The transaction and attribution succeeded but the requested transfer did not match; the runner did not label it successful. Duration 21668 ms. + +## Not completed + +- Hermes installation/runtime selection, model authentication and actual model-driven MCP run. +- Fresh external paid-API purchase and response-delivery evidence. +- Persistent hosting, scheduled operation and task recovery. +- FLOP provider/session integration; official implementation requirements remain unresolved. +- Independent user reproduction or adoption. + +## Next task + +Inspect existing accessible Hermes runtime before provisioning another. Configure an isolated profile with the supplied Meter block and one provider. Run the historical proof and wrong-amount task through Hermes itself. Record version, model, tool trace, verdict and actual usage/cost. Do not claim full Hermes integration based only on the SDK runner. + +See `docs/hermes-roadmap.md` for subsequent gates and `docs/hermes-integration.md` for commands. diff --git a/examples/agent-assurance.mjs b/examples/agent-assurance.mjs new file mode 100644 index 0000000..9e447bb --- /dev/null +++ b/examples/agent-assurance.mjs @@ -0,0 +1,64 @@ +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"; +import { readFile } from "node:fs/promises"; +import { pathToFileURL } from "node:url"; + +const allowedTools = new Set(["check_x402_endpoint", "verify_base_settlement"]); + +// A successful MCP envelope is not evidence of a successful settlement. +export function evaluateResult(tool, result, input) { + if (result.isError) return { status: "ERROR", evidence: result.content }; + let evidence; + try { + evidence = result.structuredContent ?? JSON.parse(result.content.find(item => item.type === "text")?.text ?? "null"); + } catch { return { status: "ERROR", reason: "Unparseable tool evidence" }; } + if (tool === "check_x402_endpoint") { + if (!["PASS", "WARN", "FAIL"].includes(evidence?.status)) return { status: "ERROR", reason: "Missing endpoint verdict", evidence }; + return { status: evidence.status, evidence }; + } + const proof = evidence?.proof; + if (proof?.network !== "eip155:8453" || proof?.transactionHash?.toLowerCase() !== input.transactionHash?.toLowerCase() + || typeof proof?.settlementVerified !== "boolean") { + return { status: "ERROR", reason: "Missing or mismatched settlement evidence", evidence }; + } + const attributionRequired = input.declaredBuilderCode !== undefined; + return { + status: proof.settlementVerified && (!attributionRequired || proof.builderAttribution?.verified === true) ? "PASS" : "FAIL", + attributionRequired, + evidence, + }; +} + +export async function runAssurance(mcpUrl, tool, input) { + if (!allowedTools.has(tool)) throw new Error("Only read-only Meter tools are supported"); + const endpoint = new URL(mcpUrl); + if (endpoint.username || endpoint.password || !["http:", "https:"].includes(endpoint.protocol)) throw new Error("Invalid MCP URL"); + const client = new Client({ name: "base-agent-meter-assurance", version: "1.0.0" }); + const startedAt = Date.now(); + try { + await client.connect(new StreamableHTTPClientTransport(endpoint)); + const listed = await client.listTools(); + const discovered = listed.tools.find(item => item.name === tool); + if (!discovered || discovered.annotations?.readOnlyHint !== true) throw new Error("Requested read-only tool is unavailable"); + const result = await client.callTool({ name: tool, arguments: input }, undefined, { timeout: 90_000 }); + return { checkedAt: new Date().toISOString(), durationMs: Date.now() - startedAt, mcpUrl: endpoint.href, tool, input, ...evaluateResult(tool, result, input) }; + } finally { await client.close(); } +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + const [mcpUrl, tool, inputFile] = process.argv.slice(2); + if (!mcpUrl || !tool || !inputFile) { + console.error("Usage: node examples/agent-assurance.mjs "); + process.exitCode = 2; + } else { + try { + const input = JSON.parse(await readFile(inputFile, "utf8")); + const report = await runAssurance(mcpUrl, tool, input); + console.log(JSON.stringify(report, null, 2)); + process.exitCode = ["PASS", "WARN"].includes(report.status) ? 0 : 1; + } catch (error) { + console.log(JSON.stringify({ status: "ERROR", message: error instanceof Error ? error.message : "Assurance failed" })); + process.exitCode = 1; + } + } +} diff --git a/examples/hermes/config.yaml b/examples/hermes/config.yaml new file mode 100644 index 0000000..edb635e --- /dev/null +++ b/examples/hermes/config.yaml @@ -0,0 +1,9 @@ +# Merge this block into the isolated Hermes profile's config.yaml. +# Model/provider settings are configured separately with Hermes setup. +mcp_servers: + base_meter: + url: "https://base-receipt-six.vercel.app/meter/mcp" + tools: + include: + - check_x402_endpoint + - verify_base_settlement diff --git a/examples/hermes/historical-receipt.json b/examples/hermes/historical-receipt.json new file mode 100644 index 0000000..3b5e794 --- /dev/null +++ b/examples/hermes/historical-receipt.json @@ -0,0 +1,7 @@ +{ + "transactionHash": "0xae5b6ab118a58aed27c89b05ef9af7e75406487d4d42496284e6767f6cf14483", + "expectedPayTo": "0x94705A9d675daa924F9190Eca4c05ED6B12d5345", + "expectedPayer": "0x94705A9d675daa924F9190Eca4c05ED6B12d5345", + "expectedAmount": "10000", + "declaredBuilderCode": "bc_87fjmj1l" +} diff --git a/test/agent-assurance.test.mjs b/test/agent-assurance.test.mjs new file mode 100644 index 0000000..fa24e11 --- /dev/null +++ b/test/agent-assurance.test.mjs @@ -0,0 +1,40 @@ +import express from "express"; +import { describe, expect, it } from "vitest"; +import { handleMeterMcpRequest } from "../src/mcp.ts"; +import { evaluateResult, runAssurance } from "../examples/agent-assurance.mjs"; + +const input = { transactionHash: `0x${"1".repeat(64)}`, expectedPayTo: "0x0000000000000000000000000000000000000001", expectedAmount: "10000", declaredBuilderCode: "seller_code" }; +const evidence = (settled, attributed) => ({ proof: { network: "eip155:8453", transactionHash: input.transactionHash, settlementVerified: settled, builderAttribution: { verified: attributed } } }); +const envelope = value => ({ content: [{ type: "text", text: JSON.stringify(value) }] }); + +describe("agent assurance evidence", () => { + it("does not mistake successful transport for successful payment or attribution", () => { + expect(evaluateResult("verify_base_settlement", envelope(evidence(false, true)), input).status).toBe("FAIL"); + expect(evaluateResult("verify_base_settlement", envelope(evidence(true, false)), input).status).toBe("FAIL"); + expect(evaluateResult("verify_base_settlement", envelope(evidence(true, true)), input).status).toBe("PASS"); + const { declaredBuilderCode, ...withoutCode } = input; + expect(evaluateResult("verify_base_settlement", envelope(evidence(true, false)), withoutCode).status).toBe("PASS"); + }); + it("rejects malformed and unrelated evidence", () => { + expect(evaluateResult("verify_base_settlement", envelope({}), input).status).toBe("ERROR"); + const wrong = evidence(true, true); wrong.proof.transactionHash = `0x${"2".repeat(64)}`; + expect(evaluateResult("verify_base_settlement", envelope(wrong), input).status).toBe("ERROR"); + expect(evaluateResult("check_x402_endpoint", { isError: true, content: [] }, {}).status).toBe("ERROR"); + expect(evaluateResult("check_x402_endpoint", envelope({ status: "WARN" }), {}).status).toBe("WARN"); + }); + it("discovers and invokes the actual HTTP MCP path, preserving failed proof", async () => { + const app = express(); app.use(express.json()); + app.all("/mcp", (req, res) => void handleMeterMcpRequest(req, res, { + rpcUrl: "https://rpc.example", verifySettlement: async () => evidence(false, false).proof, + })); + const listener = app.listen(0, "127.0.0.1"); + await new Promise(resolve => listener.once("listening", resolve)); + try { + const url = `http://127.0.0.1:${listener.address().port}/mcp`; + const report = await runAssurance(url, "verify_base_settlement", input); + expect(report.status).toBe("FAIL"); + expect(report.evidence.proof.settlementVerified).toBe(false); + await expect(runAssurance(url, "issue_base_receipt", input)).rejects.toThrow("Only read-only"); + } finally { await new Promise((resolve, reject) => listener.close(error => error ? reject(error) : resolve())); } + }); +});