MindGraph

Local-first evidence intelligence for Markdown and AI agents

Find the right source. Respect versions and permissions. Stop when evidence conflicts. Answer with citations.

English · 简体中文

Why MindGraph · Evidence guarantees · Features · Tech Stack · Quickstart · MCP · Architecture · Evaluation · FAQ

MindGraph turns a Markdown or Obsidian vault into a local, governed evidence layer for people and AI agents. It combines hybrid retrieval with source ownership, version, lifecycle, access-control, conflict and citation checks—so an answer is generated only when the evidence is fit to support it.

The first vertical is policy-heavy knowledge such as expense, finance and compliance. The same evidence pipeline can support any Markdown knowledge base where freshness, permissions and traceability matter.

Why MindGraph

The failure MindGraph is built for: Which expense deadline applies on 2026-08-18: 30 days or 60 days?

Generic RAG	MindGraph
May retrieve the semantically similar but archived 60-day policy	Filters evidence by source, status and effective date
May silently mix two active versions	Returns conflicting_evidence before calling the LLM
Produces a fluent answer with unclear provenance	Returns the source, version, date and retrieval trace

MindGraph's product principle: govern the evidence before generating the answer.

Local-first SQLite and local indexes Keep knowledge under your control	Evidence-first Citations and retrieval traces Return to the original source	Version-aware Lifecycle and effective dates Stop on conflicting evidence	Controlled Agent access REST, SSE and MCP Default read-only, gated actions

Evidence guarantees

• Source ownership before sync: register and audit a canonical directory before it can write. A note cannot silently move from one source to another, and an unsafe audit blocks the operation.
• Scoped retrieval, end to end: callers can narrow a request to selected source_ids; the same source, ACL, status and effective-date boundaries apply to base retrieval and relation expansion.
• Citations mean what they say: the response distinguishes retrieved candidate evidence from the subset actually cited in the answer body, making answer-level citation checks meaningful.
• Fail closed where integrity matters: ambiguous ownership, malformed ACLs, unsafe index shrinkage and evidence conflicts stop the unsafe path rather than being silently “fixed” by the model.

Features

Retrieval & routing

• Hybrid retrieval: BGE / FAISS dense search + BM25 sparse search + RRF fusion
• Adaptive routing: selects an appropriate retrieval strategy from query intent
• Controlled graph expansion: only human-confirmed relations can add evidence; source and ACL scopes remain in force during expansion

Evidence & governance

• Grounded answers: streaming responses with citations, retrieval traces, and a clear distinction between offered and actually cited evidence
• Policy lifecycle: stable policy_key, version, status and effective-date filtering
• Conflict-before-generation: conflicting active versions stop the LLM call
• Governed access: API key / OIDC, workspace / department ACLs, audit logs, and read-only defaults when AUTH_MODE=off
• Resilient index operations: declared corpus scope, startup consistency checks, and a guard that blocks unsafe index shrinkage

Interfaces & evaluation

• Web and Obsidian clients: ask, inspect evidence, review relations and compare runs
• Agent execution: REST/SSE, feature-gated background tasks and governed MCP tools
• Evaluation ledger: retrieval, deterministic answer trust, routing, graph gate, latency and cost in one history

Tech Stack

Layer	Technology
Runtime	CPython 3.13.15; SQLite ≥3.51.3 or approved backport 3.44.6/3.50.7
API	FastAPI (REST + SSE), MCP server
Retrieval	BGE embeddings · FAISS (dense) · BM25 (sparse) · RRF fusion
Storage	SQLite (WAL), versioned FAISS index
Clients	React/Vite web workspace (web/), Obsidian plugin
Quality	pytest, Ruff, mypy (see docs/DEPLOYMENT.md)

Quickstart

Prerequisite: CPython 3.13.15 and uv. Product WAL paths fail closed on an unapproved SQLite runtime.

Option A: verify the pipeline without keys

The public demo-vault/ contains synthetic policies, workflows and cases. This command verifies sync, indexing, hybrid retrieval, confirmed-relation expansion and ablation in a temporary directory.

git clone https://github.com/kayon0209/mindgraph.git
cd mindgraph

uv python install 3.13.15
uv sync --frozen --extra dev
uv run --frozen --no-sync python scripts/validate_mindgraph_offline.py

The offline check uses deterministic fake embeddings and a fake LLM. It proves that the engineering path is reproducible; it does not claim real-model quality.

Option B: start the Web workspace

cp .env.example .env                   # Windows: Copy-Item .env.example .env
# Configure a model provider in .env when you want real answers.
docker compose up --build

Service	URL
Web workspace	http://127.0.0.1:3000
API	http://127.0.0.1:8000
OpenAPI docs	http://127.0.0.1:8000/api/docs

Usage (MCP)

MindGraph exposes eight read-only MCP evidence tools by default:

Tool	Purpose
mindgraph_list_notes	List notes visible to the current principal
mindgraph_get_note	Read one visible note and its governance metadata
mindgraph_search	Search knowledge and return evidence
mindgraph_list_relations	List confirmed relations whose endpoints are both visible
mindgraph_evaluation_overview	Inspect evaluation-run summaries
mindgraph_get_policy_history	Inspect visible versions for one policy key
mindgraph_concept_gaps	Inspect aggregated knowledge gaps without question text
mindgraph_verify_citations	Validate citation-marker completeness, not claim support

Start the stdio server:

PYTHONPATH=src MCP_PRINCIPAL=local-user python -m mcp_server

Claude Desktop–style configuration:

{
  "mcpServers": {
    "mindgraph": {
      "command": "python",
      "args": ["-m", "mcp_server"],
      "env": {
        "PYTHONPATH": "/absolute/path/to/mindgraph/src",
        "MCP_PRINCIPAL": "local-user"
      }
    }
  }
}

MCP capability gates

mindgraph_assist is a governed answer tool and is disabled by default; enable it only with ASSIST_MCP_ENABLED=true. Three controlled write tools are present but hidden unless their independent feature flags are enabled:

• mindgraph_save_artifact requires AGENT_WRITE_TOOLS_ENABLED=true and saves an answer/evidence snapshot only to the caller's private artifact space. It does not edit knowledge-source content or publish data.
• mindgraph_submit_evidence_feedback requires AGENT_FEEDBACK_TOOL_ENABLED=true and uses an explicit preview/submit confirmation sequence.
• mindgraph_propose_relation requires AGENT_PROPOSE_RELATION_TOOL_ENABLED=true; it can create only a proposed relation, which never enters graph expansion until human confirmation.

Tool discovery and invocation both enforce the flags, authentication, ACL and audit boundary. Knowledge-source content write-back, automatic relation confirmation, and governance-resolution workflows remain deferred.

Agent Tasks

The REST task API can submit idempotent background evidence checks, retrieve task state, cancel a queued/running task, and fetch private artifacts owned by the submitting principal. It is disabled by default and requires AGENT_TASKS_ENABLED=true; execution additionally requires an explicitly enabled worker.

Tasks operate on already governed evidence. A directory-root task is not a directory-import mechanism in this release: it fails closed with source_registration_required. Administrators must import a directory through the connector's source-registration and clean-audit workflow instead.

Architecture

flowchart LR
    A[Markdown / Obsidian] --> B[Parse, clean and govern]
    B --> C[(SQLite WAL)]
    B --> D[Versioned FAISS index]
    Q[Person / AI agent] --> R[Adaptive retrieval router]
    C --> R
    D --> R
    R --> F[Dense + Sparse + RRF]
    F --> G{Evidence conflict?}
    G -- Yes --> H[Stop and surface versions]
    G -- No --> I[Confirmed relation expansion]
    I --> J[LLM generation]
    J --> K[Answer + Citation + Trace]

Loading

Status, effective-date, category and permission filters are shared by base retrieval and relation expansion. This prevents an archived document from re-entering a current answer through a graph edge.

Repository layout

src/
  api/            FastAPI routes, auth, OIDC, middleware, MCP mount
  application/    Application services (orchestration, chat, lifecycle)
  domain/         Stable models, errors and interfaces
  infrastructure/ Adapters: SQLite, parsers, model SDKs
  retrieval/      Retrieval stages: embeddings, dense, sparse, fusion, pipeline
  ui/             Streamlit client (api_client.py is the only backend entry)
evaluation/       Golden dataset, retrieval/answer/routing/ablation evaluation
demo-vault/       Public synthetic policies, workflows and cases
docs/             Architecture, deployment, product strategy and ADRs
scripts/          Validation, evaluation and ingestion utilities
tests/            Regression and contract tests (pytest)
web/              Web workspace
obsidian-plugin/  Obsidian client

Evaluation and boundaries

python scripts/run_ablation.py
python scripts/run_routing_evaluation.py
python scripts/run_answer_evaluation.py --live --strategy hybrid

The current frozen set (mindgraph_golden_v2.jsonl, version 2.4.0) contains 90 approved cases derived from the synthetic demo vault and public handbooks. It covers replacement, thresholds, exceptions, cross-policy questions, multi-condition cases, exact facts, graph-needed controls, ACL-restricted cases, no-answer cases, synonym/abbreviation and ambiguity. Retrieval reports Recall@K, Precision@K, MRR and nDCG@K with per-query_type/difficulty stratification; the deterministic-answer-v2 evaluator separately measures offered-evidence and actually-cited-evidence fidelity, refusal correctness, version validity, required facts, forbidden facts, ACL leakage, conflict accuracy, latency, tokens and estimated cost. These are local development/regression measurements, not production benchmark claims.

What MindGraph is today

• A local-first Hybrid RAG system with controlled, human-confirmed relation expansion
• Version-aware, citation-first, source-scoped and MCP-ready
• Reproducible with a public synthetic vault and deterministic offline checks

What it is not yet

• A complete entity-disambiguation and multi-hop knowledge-graph engine
• A hosted enterprise SaaS or a production-certified benchmark; the current 90 cases are local development/regression tests

Project status

Current capability	Next focus	Later exploration
Hybrid retrieval, adaptive routing and source-scoped evidence	More representative, independently reviewed evaluation coverage	Typed policy edges
Version-conflict interception, source ownership audit and index integrity guards	Stronger Ruff, mypy and coverage gates	Entity-event dual graph
ACL, OIDC, audit and default read-only access	Human-reviewed evidence feedback and controlled write workflows	Community discovery
Web, Obsidian and MCP capability gates	Connector and ACL coverage validated with enterprise deployments	Multi-hop reasoning

See docs/PRODUCT_STRATEGY.md for the product boundary and roadmap.

Source ownership for directory sync (schema v16)

The administrator directory endpoint now defaults to dry-run. It registers and validates the canonical source, then records an ownership audit; it does not create, update, prune, index, or ACL-backfill notes. Use dry_run=false only after a clean audit for the same connector and source. The full source set is preflighted before a real sync, and a stable note ID from one source cannot take over a note owned by another. Unknown ownership, overlapping roots, disabled sources, and malformed ACL are fail-closed findings. A directory-root task is not an import path in this release.

FAQ

Does MindGraph require a model provider or API key?

No. The offline validation path (Option A) runs with deterministic fake embeddings and a fake LLM. You only configure a model provider in .env when you want real, grounded answers.

Why is MCP read-only by default?

MindGraph treats evidence as governed data. The eight base tools are read-only. A small set of scoped write actions can be enabled independently, with explicit confirmation, authentication, ACL and audit checks; they cannot write back to knowledge-source content or auto-confirm a relation.

How does MindGraph differ from a plain RAG pipeline?

It adds governance before generation: version/effective-date filters, ACL enforcement, conflict interception, and citations with retrieval traces. See Why MindGraph for the failure scenario it addresses.

Where does the evaluation data come from?

The frozen golden set (2.4.0, 90 cases) is derived from the public synthetic demo-vault/ and public handbooks. Metrics are local development/regression measurements, not production benchmark claims.

Documentation

• Product strategy and roadmap
• Architecture
• Deployment
• Retrieval cost and efficiency
• Obsidian plugin

Contributing

Issues, reproducible evaluation cases, documentation improvements and small focused pull requests are welcome. Especially useful contributions include real-but-shareable policy cases, MCP client examples, Obsidian workflows and retrieval or access-control boundary tests.

If evidence-first local knowledge systems are useful to you, consider giving MindGraph a star.

Report an issue · Read the roadmap · MIT License