Skip to content

feat(mcp): add hosted httpSMS server - #990

Open
AchoArnold wants to merge 33 commits into
mainfrom
feat/mcp-server
Open

AchoArnold wants to merge 33 commits into
mainfrom
feat/mcp-server

Conversation

@AchoArnold

Copy link
Copy Markdown
Member

Summary

  • add a hosted, stateless MCP server using the official Go SDK with 2026-07-28 and 2025-11-25 compatibility
  • add Firebase-backed OAuth 2.1, scoped audience-bound JWT delegation, Redis state/rate limits, and seven httpSMS tools
  • add the authenticated incoming-message API, Cloud Run deployment assets, CI gates, and full-stack integration coverage

Tools

  • list_phones
  • send_sms
  • list_message_threads
  • list_thread_messages
  • list_incoming_messages
  • create_phone_api_key
  • rotate_user_api_key

Validation

  • API unit and full suites
  • MCP unit suite and server build
  • full Docker integration stack: 44 tests before merging main, 53 tests after merging main's adapter-emulator changes
  • final targeted auth/cache review and validation

AchoArnold and others added 26 commits September 3, 2026 19:56
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add delegated MCP-to-API JWT authentication: a route-, scope-, and
subject-bound token verifier (pkg/auth) and a Fiber middleware
(MCPDelegationAuth) that loads the existing user auth context from the
token's Firebase-UID subject when the token is valid for the exact
method/path/scope of the request.

MCPDelegationAuth is registered before BearerAuth so a cryptographically
valid but insufficiently-scoped or misbound MCP token is rejected with
403 directly, instead of falling through to Firebase ID token
verification. BearerAuth now short-circuits when a prior middleware has
already populated the auth context, and no longer logs the raw bearer
token on verification failure.

The verifier is wired into the DI container from MCP_AUTH_ISSUER,
MCP_AUTH_AUDIENCE, and MCP_AUTH_JWKS_URL; it is disabled when all three
are empty and container construction fails fast when only some are set.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Add GET /v1/messages/incoming, scoped to messages:read, that reuses
MessageService.SearchMessages while forcing types=[mobile-originated].
The endpoint has no CAPTCHA requirement, unlike /v1/messages/search
which remains unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Bootstrap the mcp/ Go module: validated config.Load(), RSA KeySet
signing/JWKS (auth.NewKeySet/SignMCPAccessToken/SignAPIDelegationToken/
JWKS), and an observability.New() logging/tracing bootstrap.

API delegation tokens carry JSON fields scopes, http_method, and
http_path, matching api/pkg/auth.MCPClaims exactly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Review round 1 fixes for Task 3:

- Remove the unused firebase.google.com/go dependency and its entire
  transitive graph. It pulled a Go 1.26 floor via a transitive dep while
  this module must stay on Go 1.25; the approved Task 5 design uses a
  custom Firebase certificate/JWT verifier instead of the Admin SDK.
  go.mod remains `go 1.25.0`; the still-needed-later pins
  (modelcontextprotocol/go-sdk v1.7.0, redis/go-redis/v9, otelhttp) are
  preserved per the multi-task plan.
- Replace KeySet's exported, freely-mutable Issuer/MCPAudience/APIAudience
  fields with a private atomic.Pointer[keySetConfig] published exactly
  once via a new Configure(issuer, mcpAudience, apiAudience string) error.
  Configure rejects empty values and a second call; signing methods fail
  closed until Configure has succeeded. Publishing the whole config behind
  a single CompareAndSwap (rather than a bool flag written before the
  fields) avoids a visibility race where a reader could see "configured"
  before the fields were set.
- Add tests: unconfigured signing rejected, successful configuration,
  reconfiguration rejection (with slot-not-consumed-by-invalid-call and
  original-values-preserved checks), and a concurrent-Configure-calls test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
- Pin the CIMD document fetch's actual TCP connection to the single
  IP address already validated as public (resolve exactly once),
  instead of letting the http.Transport dial its own second,
  unvalidated DNS resolution of the client_id host. Closes a
  DNS-rebinding/TOCTOU gap. Host header and TLS SNI still use the
  original hostname since only the dial address changes.
- Reject CIMD responses whose Content-Type is not application/json
  (charset and other parameters are still allowed), wrapping
  ErrClientMetadataInvalid.
- NewRedisStore now panics for a *redis.ClusterClient or *redis.Ring:
  RotateRefreshToken's Lua script touches two independently-hashed
  keys in one atomic EVAL, which the approved key format cannot
  guarantee share a Redis Cluster hash slot. This service requires a
  standalone Redis client (redis.NewClient); documented in RedisStore.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Add multi-stage Dockerfile (Go 1.25 alpine builder, non-root alpine 3.22 runtime with ca-certificates/tzdata), cloudbuild.yaml mirroring api/cloudbuild.yaml for the http-sms-mcp Cloud Run service, .dockerignore, and operator README covering env vars, local startup, health/MCP routes, Cloud Build invocation, one-time mcp.httpsms.com domain mapping, Firebase authorized-domain setup, API coordination, signing-key rotation, standalone-Redis constraint, 2025-11-25 removal plan, and secret/redaction rules.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
# Conflicts:
#	tests/README.md
#	tests/docker-compose.yml
Delegated MCP verification now prefilters bearer tokens on their
unverified "iss" claim -- parsed only to discard tokens that cannot be
ours and never trusted for authentication -- so Firebase ID tokens and
other non-MCP credentials seen by the pre-BearerAuth middleware never
reach the JWKS cache. The JWKS cache itself collapses concurrent
refreshes into one in-flight fetch, refuses a new fetch until
MinRefreshInterval (default one minute) has elapsed, and keeps serving
an already known key while throttled, mirroring the MCP Firebase
certificate cache. The 2s HTTP timeout, key rotation, and fail-open
middleware behavior are unchanged.

The MCP CIMD client cache is bounded at 1024 entries, purging expired
entries and then deterministically evicting the entry closest to
expiring under the existing mutex, preserving its 15-minute TTL.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
@gitguardian

gitguardian Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
36901872 Triggered RSA Private Key bf06243 mcp/internal/config/config_test.go View secret
36901871 Triggered Generic High Entropy Secret a13bc7d mcp/internal/tools/api_keys_test.go View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@codacy-production

codacy-production Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 11 critical · 21 high · 16 medium · 52 minor

Alerts:
⚠ 100 issues (≤ 0 issues of at least minor severity)

Results:
100 new issues

Category Results
Security 2 minor
21 high
11 critical
16 medium
CodeStyle 50 minor

View in Codacy

🟢 Metrics 1913 complexity · 840 duplication

Metric Results
Complexity 1913
Duplication 840

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Comment thread mcp/internal/oauth/clients.go Dismissed
Comment thread mcp/internal/oauth/authorize.go Dismissed
@greptile-apps

greptile-apps Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds a standalone, stateless MCP service with Firebase-backed OAuth, scoped delegated JWT authentication, Redis state and rate limits, seven httpSMS tools, and a dedicated incoming-message API.

  • Adds OAuth metadata, authorization-code and refresh-token grants, client registration, signing keys, and JWKS publication.
  • Adds API-side verification for audience-, scope-, method-, and path-bound MCP delegation tokens.
  • Adds MCP tool handlers and a typed httpSMS API client for phones, messages, threads, and API-key operations.
  • Adds Cloud Run assets and expanded unit and full-stack integration coverage.

Confidence Score: 4/5

The PR is not ready to merge because the checked-in production deployment leaves API-side MCP delegation disabled, causing all hosted MCP tool calls to fail authorization.

The MCP service correctly mints delegated API tokens, but the production API deployment supplies none of the trust settings required to install their verifier, so those tokens cannot authenticate against api.httpsms.com.

Files Needing Attention: api/pkg/di/config.go, api/pkg/di/container.go, api/cloudbuild.yaml

Important Files Changed

Filename Overview
api/pkg/di/config.go Introduces optional MCP verifier configuration whose disabled-by-default behavior leaves production delegation inactive without coordinated deployment settings.
api/pkg/di/container.go Installs delegated authentication before Firebase bearer authentication, but only when the missing production trust configuration enables it.
api/pkg/auth/mcp_token_verifier.go Adds bounded RS256 verification with issuer, audience, scope, method, and exact request-path enforcement.
api/pkg/handlers/message_handler.go Adds the authenticated incoming-message endpoint and forces searches to mobile-originated messages.
mcp/internal/oauth/token.go Implements PKCE-bound authorization-code exchange and scope-narrowing refresh-token rotation with atomic Redis state transitions.
mcp/internal/oauth/clients.go Implements bounded client metadata retrieval with public-address validation, DNS pinning, redirect rejection, and exact client-ID binding.
mcp/internal/tools/api_keys.go Adds scoped API-key tools with one-time, user/client/operation-bound confirmation before primary-key rotation.
mcp/internal/tools/messages.go Adds scoped message and thread tools with bounded schemas and direct mapping to operation-bound API calls.
mcp/cloudbuild.yaml Deploys the hosted MCP service and its secrets, but cannot by itself establish the required trust configuration on the API service.
api/cloudbuild.yaml Continues deploying the API without the MCP issuer, audience, or JWKS variables required for the newly added delegated authentication path.

Sequence Diagram

sequenceDiagram
  participant Client as MCP Client
  participant MCP as Hosted MCP Server
  participant Firebase as Firebase
  participant Redis as Redis
  participant API as httpSMS API
  Client->>MCP: OAuth authorization + PKCE
  MCP->>Firebase: Verify identity token
  MCP->>Redis: Store/consume grant state
  MCP-->>Client: MCP access + refresh tokens
  Client->>MCP: Authenticated tool call
  MCP->>MCP: Check scope and mint operation-bound JWT
  MCP->>API: HTTP request with delegated JWT
  API->>MCP: User-scoped result
  MCP-->>Client: Structured tool result
Loading

Reviews (1): Last reviewed commit: "fix(auth): bound token metadata caches" | Re-trigger Greptile

Comment thread api/pkg/di/config.go
Filter mobile-originated messages in the MCP tool instead of adding a dedicated API route.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Configure API trust for hosted MCP delegation tokens and replace static secret-like test fixtures to prevent scanner noise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
@AchoArnold

Copy link
Copy Markdown
Member Author

Secret-scan follow-up: both GitGuardian findings were test-only fixtures, not deployable credentials. Commit c2556b3 removes the checked-in throwaway RSA private-key fixture by generating it at test runtime and replaces the high-entropy-looking API-key test value. No production secret was exposed or requires rotation.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Production Firebase verification is currently broken, with additional unresolved OAuth security and tool correctness issues.

Review effort: Balanced
Findings: 3 High severity · 2 Medium severity

Open (5)
What changed in this PR

Adds a hosted MCP service with OAuth authentication, delegated API access, Redis-backed state/rate limits, deployment assets, and integration coverage.

Changes:

  • Implements seven MCP tools and OAuth 2.1 flows.
  • Adds API-side delegated JWT authentication.
  • Adds Cloud Run, Docker, CI, and integration-test support.
File Description
tests/​seed.sql Adds MCP fixtures.
tests/​integration_test.go Makes key rotation rerunnable.
tests/​helpers_test.go Cleans up webhooks.
tests/​go.sum Updates test checksums.
tests/​go.mod Adds MCP test dependencies.
tests/​generate-firebase-credentials.sh Generates test signing assets.
tests/​docker-compose.yml Adds MCP test service.
tests/​.gitignore Ignores generated credentials.
tests/​.env.test Configures delegated authentication.
README.md Extends integration timeout.
mcp/​internal/​tools/​register.go Registers MCP tools.
mcp/​internal/​tools/​phones.go Implements phone listing.
mcp/​internal/​server/​stream_internal_test.go Tests streaming flush behavior.
mcp/​internal/​server/​rate_limit.go Implements Redis rate limits.
mcp/​internal/​server/​rate_limit_test.go Tests rate limiting.
mcp/​internal/​observability/​observability.go Configures logs and tracing.
mcp/​internal/​oauth/​token.go Implements token grants.
mcp/​internal/​oauth/​templates/​authorize.html Adds hosted consent UI.
mcp/​internal/​oauth/​metadata.go Serves OAuth metadata.
mcp/​internal/​oauth/​metadata_test.go Tests metadata documents.
mcp/​internal/​oauth/​clients_cache_test.go Tests client cache bounds.
mcp/​internal/​httpsms/​transport.go Redacts traced query strings.
mcp/​internal/​httpsms/​models.go Defines API client models.
mcp/​internal/​config/​config.go Loads MCP configuration.
mcp/​internal/​config/​config_test.go Tests configuration validation.
mcp/​internal/​auth/​middleware.go Verifies MCP bearer tokens.
mcp/​internal/​auth/​middleware_test.go Tests MCP authentication.
mcp/​internal/​auth/​keys.go Mints and verifies JWTs.
mcp/​internal/​auth/​firebase.go Verifies Firebase tokens.
mcp/​internal/​auth/​claims.go Defines delegated claims.
mcp/​go.mod Defines MCP module dependencies.
mcp/​Dockerfile Builds the MCP image.
mcp/​cmd/​server/​main.go Assembles and runs service.
mcp/​cmd/​server/​main_test.go Tests startup and shutdown.
mcp/​cloudbuild.yaml Deploys MCP to Cloud Run.
mcp/​.dockerignore Restricts build context.
api/​pkg/​middlewares/​mcp_delegation_auth_middleware.go Authenticates delegated JWTs.
api/​pkg/​middlewares/​mcp_delegation_auth_middleware_test.go Tests delegated middleware.
api/​pkg/​middlewares/​bearer_auth_middleware.go Preserves prior authentication.
api/​pkg/​middlewares/​bearer_auth_middleware_test.go Tests auth short-circuiting.
api/​pkg/​di/​container.go Wires delegated authentication.
api/​pkg/​di/​config.go Loads delegation settings.
api/​pkg/​di/​config_test.go Tests delegation configuration.
api/​pkg/​auth/​mcp_token_verifier.go Validates delegated tokens.
api/​pkg/​auth/​mcp_token_verifier_test.go Tests token validation.
api/​pkg/​auth/​mcp_jwks.go Fetches and caches JWKS.
api/​pkg/​auth/​mcp_jwks_test.go Tests JWKS caching.
api/​pkg/​auth/​mcp_claims.go Defines API-side claims.
.github/​workflows/​api.yml Adds MCP CI validation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread mcp/internal/config/config.go Outdated
Comment thread mcp/internal/oauth/clients.go
Comment thread mcp/internal/oauth/store.go Outdated
Comment thread mcp/internal/httpsms/models.go
Comment thread mcp/internal/tools/messages.go Outdated
Bound unauthenticated client registration, revoke refresh-token families on reuse, and restore server-side incoming-message pagination. Align Firebase certificate parsing and preserve contact details in tool output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Transport failures can expose sensitive query values, and several MCP tools silently accept invalid negative pagination.

Review effort: Balanced
Findings: 1 High severity · 1 Low severity

Open (2)
Resolved since last review (5)

Comment thread mcp/internal/httpsms/client.go
Comment thread api/pkg/handlers/message_handler.go Outdated
Export correlated MCP traces and redacted structured logs to Axiom through SDK receiving and sending middleware.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9ff1e38a-b018-4cf7-a5e9-5044a2efd03c
Strip sensitive query values from wrapped URL errors and publish the incoming owners filter as optional.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 068b4956-94f1-4bbf-af3f-809a3a53d43e
Prevent notification scheduling from racing phone SENT events in the integration suite.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 068b4956-94f1-4bbf-af3f-809a3a53d43e

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants