Continuous Penetration Testing as a Managed Subscription
AWS MSP AI Hackathon — Track 2: Deliver AI-Driven Outcomes for Customers
Target Special Award: Best AWS Frontier Agents Application (AWS Security Agent)
⚠ The first real Continuum test happens on 23 November 2026 or later — tracked in issue #36 (label
blocker-submission). Until that test is done and its redacted evidence is indocs/preuves/, the "the code really calls an AWS AI service at runtime" requirement is not proven, and the Devpost submission (deadline 11 December 2026, 12:00 PT) cannot be sent. The single procedure isdocs/runbook-premier-test.md: eligibility first (Billing → Free Tier + Continuum console), evidence sheet, thenfree_tier_confirmedon the tenant, then one test. NoStartPentestJobbefore that gate;REMPART_FREE_TIER_ONLY=1stays on.Everything else is live today: the public judge demo on GitHub Pages (0 €, frozen dataset), the AWS-backed portal on CloudFront, the API, and the CI verdict under 5 minutes. The full honest status list is
docs/submission-checklist.md.
Traditional penetration testing happens once a year, producing an obsolete PDF weeks after execution while software development teams ship code daily.
Rempart enables Managed Service Providers (MSPs) to deliver continuous application penetration testing as a recurring monthly subscription:
- Continuous CI/CD Gating: At every release, AWS Security Agent (Amazon Continuum) automatically tests deployed increments in staging, blocking promotion if P1/P2 vulnerabilities are uncovered.
- Contextual AI Triage: A Strands triage agent backed by Amazon Bedrock (
BedrockModel) scores findings against business exposure, applying deterministic guardrails to prevent hallucinated severity or unlawful dismissals. - Closed-Loop Remediation: Security Agent generates automated remediation pull requests; post-merge retests cryptographically prove vulnerability closure before closing tickets.
- Multi-Tenant Fleet Governance: Central serverless control plane managing distributed customer accounts, enforcing signed test mandates, verified domains, and task-hour budget caps ($50/hour rate).
The model provider is chosen by one variable, REMPART_LLM_PROVIDER: bedrock in the deployed environment (the Lambda role holds bedrock:InvokeModel on the chosen model ARNs, nothing wider), openai_compat as the local fallback. The guardrails — deterministic priority matrix, mandatory human validation on Critical/High, prompt-injection isolation — are identical for both, and a monthly LLM budget ceiling falls back to deterministic triage instead of letting the invoice run away. See HLD § 8.
Detailed architectural specifications: High-Level Design (HLD) | French Functional Specification (Cahier des charges).
Live, free, no AWS account, no cost: https://tradoooo.github.io/rempart/ — the portal runs in offline demo mode on a frozen dataset (0 €, zero customer data on a public URL).
- Open
/fleet— the MSP cockpit: two demo tenants, headline KPIs, findings by priority. - Look at
/evidence— the sealed evidence dossier with a SHA-256 fingerprint a third party can recompute. - Play the
/triagequeue — the LLM triage proposal (P1 priority, 7-day deadline) and the guardrail that forces a Critical finding to a human decision.
Nothing to install, nothing to sign, nothing to pay. Full guided tour for evaluators: docs/JUDGES.md.
AWS Security Agent operates directly inside each customer account to satisfy hard AWS quotas (20 integrations per region, 5 concurrent tests) and maintain strict billing/data isolation. Rempart orchestrates assessments remotely via cross-account IAM roles with sts:ExternalId.
flowchart LR
subgraph GIT["Customer Repository"]
REPO["Code & CI/CD Pipeline"]
end
subgraph CLIENT["Customer AWS Account"]
APP["Application (Staging / Prod)"]
SA["AWS Security Agent (Agent Space)"]
ROLE["Cross-Account IAM Role (rempart-operator)"]
end
subgraph MSP["MSP Control Plane Account"]
PORTAL["Web Portal (CloudFront + S3)"]
API["API Gateway + Lambda"]
AUTH["Amazon Cognito (RBAC)"]
SFN["AWS Step Functions (Orchestrator)"]
AGT["Strands Triage Agent (dispatch_triage Lambda)"]
BR["Amazon Bedrock (InvokeModel on chosen models)"]
DB["DynamoDB (Multi-Tenant Store)"]
end
REPO -- "Deploy" --> APP
REPO -- "CI/CD Gate" --> SA
SA -- "Assesses via Verified Domain" --> APP
SA -- "Remediation PR" --> REPO
SFN -- "AssumeRole (ExternalId)" --> ROLE
ROLE --> SA
SFN --> AGT
AGT -- "bedrock:InvokeModel" --> BR
PORTAL --> API
API --> AUTH
API --> DB
API --> SFN
All screenshots are taken from the local demo (frozen dataset, no AWS
account, no cost) — regenerate them at any time with
./scripts/capture_screenshots.sh.
| Fleet (MSP view) | Triage queue |
|---|---|
![]() |
![]() |
| Finding detail | Budget & forecast |
|---|---|
![]() |
![]() |
| Evidence dossier (scellé, empreinte recalculable) | Industrialization |
|---|---|
![]() |
![]() |
| Audit journal | Contract correction policy |
|---|---|
![]() |
![]() |
Every other screen (tenant, application, run, reports, login)
| Screen | Image |
|---|---|
| Tenant | ![]() |
| Application | ![]() |
| Run (pentest job) | ![]() |
| Monthly reports | ![]() |
| Login / demo mode | ![]() |
├── backend/
│ ├── pyproject.toml # Ruff and Pytest configuration
│ ├── rempart/
│ │ ├── common/ # Models, deterministic priority matrix, budget, authz, store
│ │ ├── api/ # HTTP Lambda handlers (Cognito JWT validation, tenant isolation)
│ │ ├── orchestrator/ # Step Functions tasks, Security Agent client, GitHub ticketing
│ │ ├── triage/ # Strands agent (Bedrock / OpenAI-compatible), guardrails, tools
│ │ └── reports/ # Deterministic KPI calculations, HTML/PDF monthly report
│ └── tests/ # Fast, zero-network test suite using test doubles (fakes.py)
├── iac/
│ ├── control-plane/ # AWS CDK Python app for MSP infrastructure
│ └── client-account/ # rempart-client.template.yaml for customer onboarding
├── portal/ # Vite + React 18 + TypeScript web dashboard (English)
├── demo/
│ ├── app-a/ # ShopLite (seeded IDOR vulnerability)
│ ├── app-b/ # BookWorm (seeded SQLi / unauthenticated admin)
│ └── gate/ # Reusable GitHub Actions CI/CD gate workflow
└── docs/
├── HLD.md # High-Level Design for hackathon submission
├── acceptance.md # Acceptance test suite & scenario matrix (R1–R12)
└── cahier-des-charges-...md # Detailed functional specification
Rempart enforces clean modular boundaries: all backend modules depend solely on rempart.common. Tests execute purely in memory without AWS or network dependencies.
# Set up Python 3.12 virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install backend dependencies
cd backend
pip install -e ".[dev]"
# Run complete acceptance & unit test suite
python -m pytest tests -vcd portal
npm install
npm run build # Validates TypeScript compilation and bundles app
npm test # Runs frontend Vitest suiteThe control plane deploys DynamoDB tables, Cognito user pools, API Gateway, Step Functions workflows, and CloudFront distributions:
cd iac/control-plane
pip install -r requirements.txt
cdk bootstrap aws://<MSP_ACCOUNT_ID>/<REGION>
cdk deploy --allCustomer provisions cross-account trust in their AWS account using iac/client-account/rempart-client.template.yaml:
aws cloudformation deploy \
--template-file iac/client-account/rempart-client.template.yaml \
--stack-name rempart-client-agent \
--capabilities CAPABILITY_NAMED_IAM \
--parameter-overrides \
MspAccountId="<MSP_ACCOUNT_ID>" \
MspOrchestratorRoleName="rempart-orchestrator" \
ExternalId="<TENANT_UNIQUE_UUID>" \
GitHubOrg="<CUSTOMER_GITHUB_ORG>" \
GitHubRepo="<CUSTOMER_GITHUB_REPO>"git clone git@github.com:Tradoooo/rempart.git && cd rempart
python3 -m venv .venv && .venv/bin/pip install -r backend/requirements-ci.txt
.venv/bin/python scripts/serve_local.py # API + frozen dataset on :8000
cd portal && npm ci && npm run dev:local # portal pointed at the local APIThe local server serves the frozen dataset (demo/seed/demo-dataset.json:
2 customers, 2 applications, 6 runs, 6 findings across the whole lifecycle,
10 analyst decisions, budgets, 2 monthly reports, 22 audit events, 2 sealed
evidence dossiers (SHA-256 fingerprint a third party can recompute),
industrialization metrics, a budget forecast and the correction policy) through
the real API handlers and the real authorisation layer — so tenant isolation is
enforced for real: a client_security token from tenant A gets 403 on every
tenant-B route. No AWS call is made, so nothing is billed.
Les deux options ci-dessous ne facturent rien : aucun compte AWS, aucun
appel sortant, aucune donnée client sur une URL publique. Ce n'est pas une
promesse mais un test : il coupe toute connexion réseau et vérifie que la
démo répond quand même
(backend/tests/scripts/test_serve_local.py::test_la_demo_locale_ne_tente_aucun_appel_reseau).
Le déploiement AWS complet (Cognito, DynamoDB, Step Functions, Security Agent) est optionnel : il démontre le produit branché, pas la démo.
https://tradoooo.github.io/rempart/
Published on GitHub Pages from the gh-pages branch by
.github/workflows/deploy-pages.yml. It runs in offline demo mode: the data
is the frozen dataset, the same one served by scripts/serve_local.py. No
backend, no AWS account, no cost, and no real customer data on a public URL —
the portal says so with an "OFFLINE DEMO MODE" banner.
The full AWS-backed deployment (Cognito, DynamoDB, Step Functions, real
Continuum calls) still needs credentials and a verified domain; tracked in
docs/submission-checklist.md.
- Portal URL:
https://tradoooo.github.io/rempart/ - Judge credentials: provided in the Devpost submission form (one read-only
Cognito account per role, provisioned on the frozen dataset; passwords are
generated, kept in the team vault and never committed) — role
demo(read-only, banner "Judge demo — read-only", one on-demand test per day on the minimal scope)
- Fleet — multi-tenant metrics: delivery-to-result latency, P1/P2 blocked before production, mean time to remediate by priority, coverage.
- Finding — the seeded IDOR on
/orders/{id}: the Strands agent's plain-language summary, priorityP1from the deterministic matrix, 7-day deadline, and thepending_validationstate that proves the model cannot settle a Critical finding alone. - Closed loop — the remediation pull request requested from AWS Continuum (formerly "AWS Security Agent"), then the retest that closes the finding with the retest job id as proof.
- Budget — task-hour consumption against the monthly cap (50 $/task-hour), alerts at 50/80/100 % and the auto-stop switch.
- Reports — the monthly report (HTML + PDF) whose numbers are identical to the dashboard, because both are computed by the same deterministic code.
This repository implements rigorous delivery and governance standards:
- 5-Minute CI Budget: every job declares
timeout-minutes: 5, andci-budgetfails the build if any job declares more. Measured: backend ~1 m, IaC ~1 m, demo 35 s, portal 22 s. - Deterministic AI (ENF-IA-01): all metrics, task-hour costs and SLA due dates are computed by Python. The model never computes a number — and after the triage agent answers, the priority is recomputed by the matrix, which wins on divergence.
- Safety Guardrail (ENF-IA-02): a Critical or High finding can never be dismissed by the model; it always waits for a human decision.
- Prompt-injection resistance (ENF-IA-03): finding content is passed as delimited data (
<finding>…</finding>) and the system prompt forbids following any instruction inside it. - Zero-cost by default:
REMPART_FREE_TIER_ONLY=1refuses any real test until free-tier eligibility is confirmed per tenant, andREMPART_MAX_TASK_HOURS_PER_TEST=5stops a runaway test. See the free-tier runbook. - 13-month retention, enforced: every write stamps
expire_at, so DynamoDB TTL actually purges findings (ENF-DATA-01). Configuring TTL without writing the attribute would purge nothing. - Auditability: an immutable audit journal (read-only API, writes refused) plus an exportable, SHA-256-sealed evidence pack for ISO 27001 / SOC 2 reviews.
- Contract-tested infrastructure: a test asserts every Lambda handler declared in CDK exists with an
(event, context)signature — this caught a placeholder that would have deployed empty functions. - Code Limits: files < 500 lines, functions < 50 lines, zero duplication — verified by an AST scan, not by intention.
AWS MSP AI Hackathon — Track 2, Deliver AI-Driven Outcomes for Customers. Submission deadline: Friday 11 December 2026, 12:00 PT.
- Devpost submission text — ready to paste into the form
- Architecture diagram (PNG) — exported from the Mermaid diagram above, for the Devpost form (
bash scripts/export_architecture_png.sh) - What Rempart adds on top of AWS Continuum — the answer to "isn't this just a wrapper?", every claim backed by a file or a test
- Free-tier runbook — the four coded locks that make a paid test impossible by default, and the zero-cost demo path
- AWS Continuum API verification — action-by-action check against the official API reference, and the pricing trap we avoided
- Submission checklist — official § 18.1 items, honest statuses and remaining blockers
- Test instructions for judges — 10-minute guided tour, demo credentials, fallback path
- Video script — 3 minutes, timed shot by shot (recording is done by the team)
- 100 ideas to continue the project — ranked roadmap, annotated with judge impact / cost / risk












