Ingest specs. Extract requirements. Validate visually. Report with confidence.
QA Vision is a fully local, privacy-first UI quality assurance platform that bridges the gap between human-readable software requirement specifications and automated visual testing. Zero data leaves your machine — every model call, every screenshot, every validation result is processed entirely on-premises.
Unlike cloud-dependent testing suites, QA Vision operates on a dual-engine validation architecture:
- Tesseract OCR + DOM BBox mapping for text-based elements (
button,text,link) - Local Vision LLM via Ollama (LLaVA or compatible) for non-text elements (
icon,image,input,search) that OCR cannot detect
The pipeline is orchestrated by a Next.js 16 App Router dashboard that communicates with a standalone TypeScript agent through a secure file-based IPC mechanism — no shared memory, no sockets, no race conditions.
Accept requirements in any format your team already uses:
- PDF — parsed with
unpdf(Edge-compatible, zero native dependencies) - DOCX — parsed with
mammoth(server-external to avoid Turbopack bundling issues) - TXT / Markdown — read directly as UTF-8 via Node.js
fs - Files are securely streamed to the backend, never stored in browser memory
A locally running Ollama LLM (llama3.1) acts as a deterministic requirements parser:
- Ignores architectural prose, philosophy text, and abstract layout descriptions
- Extracts only verifiable, discrete UI nodes:
button,text,link,input,icon,image - Enforces a max 6-word
descriptionfield — the exact label visible on screen - Outputs a strongly-typed JSON array consumed directly by the validation engine
The validation loop routes each requirement to the correct engine automatically:
| Requirement Type | Engine | Method |
|---|---|---|
text, button, link |
Tesseract OCR + DOM | BBox mapping → full-page OCR fallback |
image, icon, input, search |
Vision LLM (LLaVA) | Natural language YES/NO visual query |
- URL Mode — Fires a headless Chromium instance (Playwright), navigates to the target, waits for all images to load, captures a pixel-perfect 1920×1080 screenshot, and extracts live DOM bounding boxes
- Image Mode — Skips the browser layer entirely; loads a locally uploaded static mockup or screenshot directly for validation
Before OCR runs, every screenshot is processed by Jimp:
- Grayscale conversion and binarization for maximum text contrast
- Per-pixel RGB channel dominance analysis for color verification
- Bounding box annotation with red rectangles saved as embedded Base64 artifacts in the HTML report
Every test run produces a self-contained HTML report in agent/docs/ with:
- Pass/Fail summary badges with counts
- Per-requirement cards showing description, type, expected color & location
- Engine badge (
🔤 OCRvs👁 Vision LLM) on every card - Embedded annotated screenshot artifacts (Base64, no external file references)
- Auto-deletion of temp files after each test run
The dashboard UI is fully translated in English, Turkish, and German via next-intl. Language preference is persisted in a cookie for one year. Switch languages from the sidebar at any time — no page reload required.
┌─────────────────────────────────────────────────────────────────────┐
│ QA Vision Pipeline │
└─────────────────────────────────────────────────────────────────────┘
User uploads SRS document (PDF / DOCX / TXT / MD)
│
▼
┌─────────────────────┐
│ STAGE 01 │ Next.js API Route (dashboard/app/api/run)
│ DOCUMENT INGESTION │ unpdf / mammoth / fs.readFileSync
│ │ → raw UTF-8 string
│ │ → written to agent/temp/reqs_[ts].txt (IPC)
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ STAGE 02 │ agent/src/ai/extractor.ts
│ AI EXTRACTION │ Ollama llama3.1 (local LLM)
│ │ → Strict JSON array of typed UI requirements
│ │ [{ id, type, description, expected_color, location }]
└────────┬────────────┘
│
▼
┌─────────────────────┐
│ STAGE 03 │ agent/src/scraper/browser.ts
│ HYBRID EXECUTION │ ─── URL Mode ───────────────────────────────
│ │ Playwright Headless Chromium (1920×1080)
│ │ waitForFunction(images.every(i => i.complete))
│ │ DOM BBox extraction → page.evaluate()
│ │ Full-page screenshot
│ │
│ │ ─── Image Mode ─────────────────────────────
│ │ fs.readFileSync(imagePath) — no browser
│ │ Empty textBboxes array passed downstream
└────────┬────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ STAGE 04 — DUAL-ENGINE VALIDATION (agent/src/vision/analyzer.ts) │
│ │
│ for each requirement: │
│ │
│ if type ∈ { text, button, link } │
│ → Primary: DOM textBboxes lookup (exact / substring match) │
│ → Fallback: Tesseract OCR PSM-11 (sparse text mode) │
│ → Color: Jimp per-pixel RGB channel dominance scan │
│ → Location: Bounding-box centre vs PAGE_WIDTH/HEIGHT thirds │
│ │
│ if type ∈ { image, icon, input, search } │
│ → POST http://127.0.0.1:11434/api/generate (Ollama LLaVA) │
│ → Prompt: "Is '{description}' visible? Answer YES or NO." │
│ → Graceful fallback if Ollama unreachable (ECONNREFUSED) │
└────────┬────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────┐
│ STAGE 05 │ agent/src/reporter.ts
│ IPC & REPORTING │ Generates standalone HTML report
│ │ Embeds Base64 annotated artifacts
│ │ Saves to agent/docs/[slug]_[date]_[hash].html
│ │ Cleans agent/temp/ directory automatically
└─────────────────────┘
│
▼
Dashboard polls /api/reports → renders report grid
User clicks card → iframe viewer inside dashboard
| Package | Version | Purpose |
|---|---|---|
| Next.js | 16.2 | App Router framework |
| React | 19.2 | UI rendering |
| Tailwind CSS | v4 | Utility-first styling |
| next-intl | latest | i18n (EN / TR / DE) |
| unpdf | ^1.6.2 | Server-side PDF text extraction |
| mammoth | ^1.12.0 | Server-side DOCX text extraction |
| Package | Version | Purpose |
|---|---|---|
| TypeScript | ^6.0.3 | Strict type safety |
| ts-node | ^10.9.2 | Direct TS execution (no compile step) |
| ollama | ^0.6.3 | Ollama SDK for LLM extraction |
Node.js http |
built-in | Raw HTTP client for Vision LLM calls |
| Package | Version | Purpose |
|---|---|---|
| Playwright | ^1.60.0 | Headless Chromium, DOM BBox, screenshots |
| Tesseract.js | ^7.0.0 | OCR — PSM 11 sparse text mode |
| Jimp | ^0.22.10 | Image preprocessing, RGB pixel scanning, artifact annotation |
| Model | Engine | Role |
|---|---|---|
llama3.1 |
Ollama | SRS document → strict JSON requirement extraction |
llava |
Ollama | Screenshot → visual element verification (Vision LLM) |
Ensure the following are installed on your system before proceeding:
Node.js >= 20.x https://nodejs.org
Ollama latest https://ollama.com
Pull the required Ollama models:
ollama pull llama3.1
ollama pull llavaNote:
llavais the default Vision LLM. You can switch tomoondreamorllava:13bby editingOLLAMA_VISION_MODELat line 17 ofagent/src/vision/analyzer.ts.
git clone https://github.com/your-username/qa-vision.git
cd qa-visioncd dashboard
npm installcd ../agent
npm installcd agent
npx playwright install chromiumcd dashboard
npm run devThe dashboard will be available at http://localhost:3000.
- Open the dashboard and click New Test in the sidebar
- Paste your requirements or upload an SRS document (PDF, DOCX, TXT, MD)
- Select Test via URL (enter a live URL) or Test via Image (upload a screenshot)
- Click Run QA Test — the pipeline executes in the background
- When complete, the new report appears instantly in the Reports gallery
- Click any report card to open the full inline viewer, or open it in a new tab
cd agent
# URL mode
npx ts-node src/index.ts --url="https://example.com" --docFile="./temp/reqs.txt"
# Image mode
npx ts-node src/index.ts --image="./temp/upload_screenshot.png" --docFile="./temp/reqs.txt"CLI flags:
| Flag | Required | Description |
|---|---|---|
--url |
One of url/image | Target web URL to test |
--image |
One of url/image | Path to local screenshot or mockup |
--docFile |
✅ Always | Path to the UTF-8 requirements text file |
When both
--urland--imageare provided,--urltakes priority.
qa-vision/
│
├── agent/ # Standalone TypeScript QA engine
│ ├── src/
│ │ ├── index.ts # CLI entry point; orchestrates both pipelines
│ │ ├── reporter.ts # HTML report generator (Base64 artifacts, IPC cleanup)
│ │ │
│ │ ├── ai/
│ │ │ └── extractor.ts # Ollama LLM client; SRS → typed Requirement[]
│ │ │
│ │ ├── scraper/
│ │ │ └── browser.ts # Playwright headless capture; DOM BBox extraction
│ │ │
│ │ └── vision/
│ │ └── analyzer.ts # Dual-engine validator (Tesseract OCR + Vision LLM)
│ │
│ ├── temp/ # IPC staging area — auto-cleaned after each run
│ │ ├── reqs_[timestamp].txt # Requirements file written by dashboard API
│ │ └── upload_[timestamp].png # Uploaded screenshot (image mode)
│ │
│ ├── docs/ # Generated HTML reports (persisted)
│ │ └── [domain]_[date]_[hash].html
│ │
│ ├── output/ # Annotated bbox artifacts (embedded → deleted)
│ ├── eng.traineddata # Tesseract English language data
│ ├── tur.traineddata # Tesseract Turkish language data
│ ├── package.json
│ └── tsconfig.json
│
└── dashboard/ # Next.js 16 App Router frontend
├── app/
│ ├── layout.tsx # Root layout with NextIntlClientProvider
│ ├── page.tsx # Dashboard — New Test hero + stats bar
│ │
│ ├── reports/
│ │ └── page.tsx # Reports gallery (filter, sort, search, delete)
│ │
│ ├── how-it-works/
│ │ └── page.tsx # Pipeline architecture documentation page
│ │
│ └── api/
│ ├── run/route.ts # POST — triggers agent; handles FormData + IPC write
│ ├── reports/route.ts # GET — lists reports; DELETE — removes a report
│ ├── reports/html/ # GET — serves raw HTML report files
│ └── locale/route.ts # POST — sets locale cookie for i18n
│
├── components/
│ ├── Sidebar.tsx # Shared navigation sidebar (useTranslations, LocaleSwitcher)
│ └── LocaleSwitcher.tsx # EN · TR · DE language toggle (cookie-based)
│
├── i18n/
│ └── request.ts # next-intl server config (reads locale from cookie)
│
├── messages/
│ ├── en.json # English translations
│ ├── tr.json # Turkish translations
│ └── de.json # German translations
│
├── next.config.ts # withNextIntl plugin + serverExternalPackages
└── package.json
The Next.js dashboard and the TypeScript agent run as two completely separate processes. They communicate exclusively through the filesystem:
Dashboard (Next.js API Route) Agent (ts-node CLI)
────────────────────────────── ──────────────────────
1. Parse uploaded document 2. Agent starts
2. Write UTF-8 text to ┌──► 3. Reads reqs from file
agent/temp/reqs_[ts].txt ──────────┘ 4. Runs pipeline
3. Spawn: 5. Writes HTML to
npx ts-node src/index.ts agent/docs/[name].html
--url="..." --docFile="..." ◄── 6. Process exits (code 0)
4. await execPromise(command)
5. On success: clean agent/temp/
6. Return { success: true } to UI
This design guarantees:
- No shared state between web server and test runner
- No encoding issues with Unicode/Turkish characters in requirements
- The agent can be invoked independently from the CLI without the dashboard
{domain-slug}_{YYYY-MM-DD}_{HH-MM-SS}_{8-char-hash}.html
Examples:
www-google-com_2026-06-02_14-32-10_a3f8b210.html ← URL test
upload-1780386783050_2026-06-02_10-53-59_8dc07160.html ← Image test
Reports are parsed back by the dashboard API to extract domain, timestamp, source (url/image), and id (hash) for the gallery view.
| Feature | Status |
|---|---|
| Omni-document ingestion (PDF, DOCX, TXT, MD) | ✅ Shipped |
| Strict AI requirement extraction (LLM) | ✅ Shipped |
| Tesseract OCR + DOM BBox validation | ✅ Shipped |
| Playwright headless URL mode | ✅ Shipped |
| Static image upload mode | ✅ Shipped |
| Vision LLM routing (LLaVA) | ✅ Shipped |
| HTML report with Base64 artifacts | ✅ Shipped |
| Next.js dashboard + report gallery | ✅ Shipped |
| File-based IPC + temp auto-cleanup | ✅ Shipped |
| i18n (English / Turkish / German) | ✅ Shipped |
| Report deletion from UI | ✅ Shipped |
| Multi-step sequence testing | 🔜 Planned |
| AI-powered CSS fix suggestions | 🔜 Planned |
| Scheduled / CI-triggered runs | 🔜 Planned |
| Parallel requirement validation | 🔜 Planned |
| Report diffing (run comparison) | 🔜 Planned |
- Fork the repository and create a feature branch:
git checkout -b feat/your-feature - All code, variable names, comments, and commit messages must be written in strict English
- Run
npm run lintindashboard/before submitting - TypeScript strict mode is enforced in both
agent/anddashboard/— noanywithout justification - Open a pull request with a clear description of the change and its motivation
Distributed under the ISC License. See LICENSE for more information.
