Skip to content

About

An AI-powered retail analytics pipeline that transforms CCTV footage into real-time customer journey metrics using YOLOv8, ByteTrack, and Re-ID.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

StoreSight: Store Intelligence

An offline retail-analytics system. It turns raw in-store CCTV clips into structured visitor events, then serves live-style metrics (footfall, conversion, funnel, heatmap, anomalies) through a REST API and a terminal dashboard.

CCTV clips  ->  detection pipeline  ->  events.jsonl  ->  FastAPI + SQLite  ->  dashboard
              (YOLOv8 + ByteTrack)      (8 event types)   (metrics, funnel,     (rich, live)
                                                           heatmap, anomalies)

Features

  • Person detection and tracking: YOLOv8 (person class) with ByteTrack identities across frames.
  • Eight event types: ENTRY, EXIT, REENTRY, ZONE_ENTER, ZONE_EXIT, ZONE_DWELL, BILLING_QUEUE_JOIN, BILLING_QUEUE_ABANDON. Event times come from clip_start_utc + frame / fps.
  • Zone classification: the foot point of each bounding box is tested against the rectangles defined per camera in store_layout.json.
  • Staff exclusion: HSV analysis of the torso flags uniform colours (black, pink, navy); staff events are excluded from customer metrics and the visitor id becomes STF_*.
  • Re-identification: an appearance gallery (192-dim HSV histogram, cosine similarity) keeps identities across cameras and detects re-entry within 30 minutes.
  • POS correlation: a visitor who leaves the billing zone without a nearby POS transaction emits BILLING_QUEUE_ABANDON; the API uses the same transactions to compute conversion.
  • Idempotent ingestion API: events are validated with Pydantic (UUID v4, UTC timestamps, per-type rules), deduplicated by event_id, batched up to 500, with partial success reporting.
  • Analytics: unique visitors, conversion rate, average dwell per zone, queue depth, abandonment rate, a four-stage funnel, a zone heatmap, and anomaly detection (billing queue spike, dead zone, conversion drop vs a 7-day baseline).
  • Operability: SQLite persistence with in-memory read cache, structured JSON request logs with X-Trace-ID, /health with stale-feed detection, HTTP 503 when storage is unavailable.

Tech stack

Area Tools
Detection / tracking Ultralytics YOLOv8n, ByteTrack, OpenCV, NumPy
API FastAPI, Uvicorn, Pydantic v2, SQLite (sqlite3)
Dashboard / tooling rich, httpx, python-dateutil
Quality pytest, Ruff
Packaging Docker, Docker Compose

Repository layout

app/          FastAPI service (main, models, ingestion, metrics, funnel, anomalies, health,
              sessions, pos, state, storage)
pipeline/     Detection pipeline (detect, tracker, emit, run_detection.py, run.sh)
scripts/      ingest_events.py: replay events.jsonl into the API
dashboard.py  Live terminal dashboard
tests/        pytest suite (API, pipeline, POS parsing, staff detection)
docs/         DESIGN.md (architecture) and CHOICES.md (decisions and trade-offs)
store_layout.json            Cameras, clip start times and zone rectangles
pos_transactions.*.csv       Sample POS data (the real export is git-ignored)

Setup

Requires Python 3.11+. Docker is optional (API only).

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt    # detection + API + tests + dashboard

The API alone needs only requirements-api.txt, which is what the Docker image installs.

Data that is not in the repository

What Where Notes
CCTV clips videos/ (CAM 1.mp4 ... CAM 5.mp4) Supplied for the challenge, not redistributable.
YOLO weights yolov8n.pt Downloaded automatically by Ultralytics on first run.
Real POS export pos_transactions.csv Contains customer details, so it is git-ignored.
Pipeline output events/events.jsonl Produced by the pipeline.

Without the clips you can still run the API and the tests; see "Run the API".

Configuration

store_layout.json lists the cameras (file, clip_start_utc, is_entry_camera, is_backroom) and zones (zone_id, type, camera_id, pixel rectangle, sku_zone):

{
  "store_id": "STORE_BLR_002",
  "cameras": [
    {"camera_id": "CAM_3", "file": "CAM 3.mp4", "is_entry_camera": true,
     "is_backroom": false, "clip_start_utc": "2026-04-10T14:40:00Z"}
  ],
  "zones": [
    {"zone_id": "BILLING", "type": "checkout", "camera_id": "CAM_5",
     "x_min": 0, "y_min": 0, "x_max": 845, "y_max": 1080, "sku_zone": null}
  ]
}

The API reads a few optional environment variables, listed with defaults in .env.example: STORE_LAYOUT_PATH, POS_CSV_PATH, DB_PATH, AUTO_INGEST_JSONL and FORCE_DB_UNAVAILABLE.

Usage

1. Run the detection pipeline

python pipeline/run_detection.py \
  --videos-dir ./videos \
  --output-dir ./events \
  --store-layout ./store_layout.json \
  --pos-csv ./pos_transactions.challenge.csv

bash pipeline/run.sh runs the same thing with defaults. Options: --model (default yolov8n.pt), --confidence-threshold (default 0.4), --skip-frames (process every Nth frame, default 2; higher is faster but coarser). The result is events/events.jsonl:

{"event_id": "03b496a7-3772-4b2b-99b5-e434c8eaa875", "store_id": "STORE_BLR_002",
 "camera_id": "CAM_1", "visitor_id": "VIS_302A27", "event_type": "ZONE_ENTER",
 "timestamp": "2026-04-10T14:40:27Z", "zone_id": "SKINCARE_BACK", "dwell_ms": 0,
 "is_staff": false, "confidence": 0.838,
 "metadata": {"queue_depth": null, "sku_zone": "SKINCARE", "session_seq": 1}}

2. Run the API

Locally:

AUTO_INGEST_JSONL=events/events.jsonl uvicorn app.main:app --port 8000

Or with Docker (ingests events/events.jsonl at startup if it exists):

docker compose up --build

Interactive docs are at http://localhost:8000/docs. To load events into a running API (optionally replaying them in accelerated real time):

python scripts/ingest_events.py --events-jsonl events/events.jsonl
python scripts/ingest_events.py --events-jsonl events/events.jsonl --simulate-realtime

3. Watch the dashboard

python dashboard.py --api-url http://localhost:8000 --store-id STORE_BLR_002

4. Run the tests

pytest
ruff check .

The suite stubs Ultralytics, so it needs no GPU, weights or video.

API reference

Method and path Purpose
POST /events/ingest Ingest a JSON array of up to 500 events. Idempotent by event_id.
GET /stores/{store_id}/metrics Visitors, conversion, dwell per zone, queue depth, abandonment.
GET /stores/{store_id}/funnel Entry, zone visit, billing queue and purchase stages with drop-off.
GET /stores/{store_id}/heatmap Per-zone visit frequency (0-100) and dwell; data_confidence needs 20+ sessions.
GET /stores/{store_id}/anomalies BILLING_QUEUE_SPIKE, DEAD_ZONE, CONVERSION_DROP with suggested actions.
GET /health Service status and per-store feed lag (STALE_FEED after 10 minutes).

Ingest response (invalid events are reported per index, valid ones are still accepted):

{"accepted": 150, "duplicates": 2, "failed": 1,
 "errors": [{"index": 7, "event_id": "...", "code": "VALIDATION_ERROR", "details": []}]}

Metrics response:

{"store_id": "STORE_BLR_002", "period": "today", "unique_visitors": 10, "total_entries": 10,
 "conversion_rate": 0.0, "avg_dwell_ms": {"FRAGRANCE": 7540}, "queue_depth": {"current": 0},
 "abandonment_rate": 0.0, "data_from_events": 197}

Errors use a consistent envelope, for example {"error": {"code": "DB_UNAVAILABLE", "message": "Database unavailable, retry later"}} (HTTP 503).

Design notes

  • Customer metrics use the most recent day of non-staff events for the requested store.
  • A visitor counts as converted when their billing interval overlaps a POS transaction (5-minute window); each transaction converts at most one visitor.
  • Entry and exit on the entry camera use appearance mode (a new track is an entry, a lost track is an exit) because the camera angle makes line crossing unreliable. Line-crossing with hysteresis and debounce is supported through entry_line_x / entry_line_y.
  • Re-ID uses colour histograms. It runs on CPU with no extra weights, but it can confuse similarly dressed people, so cross-camera visitor counts are approximate.

More detail: docs/DESIGN.md for the architecture and docs/CHOICES.md for the decisions and trade-offs.

License

MIT, see LICENSE. The CCTV footage used during development belongs to the challenge organisers and is not included in this repository.

About

An AI-powered retail analytics pipeline that transforms CCTV footage into real-time customer journey metrics using YOLOv8, ByteTrack, and Re-ID.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages