Open Context Graph

Open-source, self-hostable enterprise context layer fabric — exposed as MCP, CLI, and Retrieval API.

Open Context Graph (openCG) is a dockerized set of services that lets a single person or an organization run a private knowledge catalogue and serve it to any AI tool — Claude, Cursor, open-weight models, in-house agents — through a single MCP endpoint. It implements the contextual-layer pipeline (identity → intent → hybrid retrieval → catalog/graph enrichment → rerank + token-budget compression → entitlement → audit) and is built to learn continuously from usage. Open-source components are preferred; local Ollama models are the default embeddings backend, with light hosted chat models reserved for intent and extraction paths.

"If your AI architecture is just your LLM tool RAG-ing against a vector database, you don't have an AI architecture. You have one component."

openCG builds out the missing components: identity, intent, hybrid retrieval, catalog/graph enrichment, rerank + compression, entitlement, audit, continuous enrichment, and an inspection UI. Every stage is independently testable, observable, and replaceable.

Version 0.1.0, released 2026-11-10: the first public release, prepared by prepare-open-source-release. Release notes are in CHANGELOG.md and on the releases page.

The thin MVP, the first end-to-end working slice, runs in Docker Compose, with an admin UI for vector search, entity inspection, ingestion control and knowledge-graph review. The full v0 vision is broader and is delivered as a series of OpenSpec changes, each going proposal, then design and specs, then tasks, implementation and archive.

Shipped (archived under openspec/changes/archive/), with the date each was archived:

In flight (openspec list):

• add-foundation is the full v0 vision, spanning 12 capabilities: intent classifier, rerank and compression, OPA enforcement, more connectors, enrichment workers, the Tempo/Grafana stack, and more. Each deferred capability ships as its own follow-up change that modifies the requirements earlier changes set.

# 1. Bring up the stack (postgres+AGE, qdrant, valkey, ollama, pipeline,
#    mcp-server, ingestion, admin-ui). First start pulls images + the embedding model.
cp .env.example .env
make up-d

# 2. Ingest a public git repo
make ingest-git REPO=https://github.com/anthropics/anthropic-cookbook

# 3. Retrieve context directly from the pipeline
curl -sS -X POST http://localhost:8000/retrieve \
  -H 'content-type: application/json' \
  -d '{
    "correlation_id":"hello-001",
    "identity":{"principal":"local-dev","roles":["admin","reader"],"tenant":"default"},
    "tool":"retrieve_for_context",
    "query":"how do I do prompt caching",
    "token_budget":2000
  }' | jq

# 4. Open the admin UI
open http://localhost:3000

Every port binds to 127.0.0.1 and can be moved in .env if it collides with something already running (see docs/OPERATIONS.md). To work on the code, start with CONTRIBUTING.md and make bootstrap.

Run the deterministic full-stack verification with make smoke. It builds a temporary local Git fixture, uses local embeddings plus an Ollama-compatible chat stub, verifies OTel spans, and enforces a five-minute deadline. A real-provider canary is opt-in via make smoke-cloud and is capped at one document and two chunks.

With the stack running (make up-d), openCG exposes its catalogue over the Model Context Protocol via the opencg MCP server — over stdio (spawn it as a subprocess) or a remote HTTP endpoint (connect by URL). Two tools are live today — opencg/retrieve_for_context (hybrid retrieval) and opencg/traverse_graph (walk the knowledge graph); the other three return not_implemented_in_mvp. Any MCP-capable client can connect.

The portable way to launch it spawns the already-built compose image on openCG's Docker network — no local Node, build, or absolute paths required:

docker run -i --rm --network opencg_default \
  -e OPENCG__PIPELINE__URL=http://pipeline:8000 \
  opencg/mcp-server:local

It only works while the stack is up — the server reaches the pipeline over the opencg_default Docker network (the compose project is named opencg, so the network name is stable). Stack down → the client shows a connection error.

claude mcp add opencg -- \
  docker run -i --rm --network opencg_default \
  -e OPENCG__PIPELINE__URL=http://pipeline:8000 opencg/mcp-server:local

Append --scope user to make it available in every project. Verify with claude mcp get opencg (expect ✓ Connected), then ask, e.g., "use opencg to find what the catalogue says about core banking."

Add to claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/claude_desktop_config.json; Windows: %APPDATA%\Claude\claude_desktop_config.json), then restart Claude Desktop:

{
  "mcpServers": {
    "opencg": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "--network", "opencg_default",
               "-e", "OPENCG__PIPELINE__URL=http://pipeline:8000",
               "opencg/mcp-server:local"]
    }
  }
}

Add the same entry to .cursor/mcp.json (this project) or ~/.cursor/mcp.json (all projects), then reload Cursor — it uses the identical mcpServers schema:

{
  "mcpServers": {
    "opencg": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "--network", "opencg_default",
               "-e", "OPENCG__PIPELINE__URL=http://pipeline:8000",
               "opencg/mcp-server:local"]
    }
  }
}

The server also speaks the Streamable HTTP transport at http://localhost:8181/mcp (the same port as the health checks) — so clients connect by URL with no subprocess, and it's reachable across the network once you expose the port (e.g. a tunnel).

Set a token before exposing it. With OPENCG__MCP__HTTP_TOKEN unset, /mcp is open (fine on loopback — the server logs an "unauthenticated" warning). Set it in .env and pass it as a bearer header to require auth.

Claude Code:

claude mcp add --transport http opencg http://localhost:8181/mcp \
  --header "Authorization: Bearer $OPENCG__MCP__HTTP_TOKEN"

(Drop --header when no token is set.) Cursor and Claude Desktop use the url form:

{
  "mcpServers": {
    "opencg": {
      "url": "http://localhost:8181/mcp",
      "headers": { "Authorization": "Bearer YOUR_TOKEN" }
    }
  }
}

claude.ai and Claude Desktop's "Add custom connector" GUI authenticate via OAuth, not a bearer header. The MCP server ships a temporary, minimal OAuth server (single shared password, in-memory tokens — see services/mcp-server/src/oauth.ts, marked REPLACE-BEFORE-PROD). Enable it by setting both:

OPENCG__MCP__PUBLIC_URL=https://your-host.ngrok.app   # externally reachable base URL
OPENCG__MCP__OAUTH_PASSWORD=<a shared access password>

Restart the mcp-server, then in claude.ai: Settings → Connectors → Add custom connector → URL https://your-host.ngrok.app/mcp. Claude runs the OAuth flow; on the consent screen, enter the access password. (The static OPENCG__MCP__HTTP_TOKEN keeps working alongside this for Claude Code / Cursor.)

This OAuth is a stopgap, not production auth: one shared password (no per-user identity), tokens are in-memory (lost on restart), self-issued. Replace with a real identity provider before any real deployment.

To avoid a container per session, build the server once and point it at the host-exposed pipeline:

pnpm install && pnpm --filter @opencg/mcp-server build

Then use node /absolute/path/to/opencg/services/mcp-server/dist/index.js as the command, with env OPENCG__PIPELINE__URL=http://localhost:8000 and OPENCG__MCP__PORT=8199 (any free port — it's only the health endpoint).

Identity is stubbed in v0 (local-dev / roles admin,reader / tenant default). Override per client with the OPENCG__IDENTITY__PRINCIPAL, OPENCG__IDENTITY__ROLES, and OPENCG__TENANT environment variables.

Without a password the UI shows an "Unauthenticated mode" banner on every page; see docs/OPERATIONS.md.

Every shared component has a Storybook story neighbour.

• TypeScript MCP server (services/mcp-server) advertising five tools. opencg/retrieve_for_context and opencg/traverse_graph are live; the other three return not_implemented_in_mvp.
• Python retrieval pipeline (services/pipeline) with four stages: identity → hybrid retrieval (Qdrant dense + Postgres FTS lexical, fused via RRF) → assemble (token budget + role→classification allow-list) → return.
• Python ingestion (services/ingestion) with a opencg-ingest git <url> CLI: clone, chunk, embed, write Postgres + Qdrant, derive code relationships with graphifyy, and extract candidate text relationships with the configured chat model.
• Next.js admin UI (services/admin-ui) with query, vector, content, ingestion, and graph-review pages.
• Storage: Postgres 16 + Apache AGE for catalog, audit, and the named knowledge graph; Qdrant for vectors; Valkey for cache; compose-internal Ollama for embeddings.
• Observability: per-stage OTel spans carrying token attributes; Prometheus counters at the pipeline's /metrics; a compose-local OTel Collector that accepts and logs development traces.
• Immutable audit log at opencg.audit_log (range-partitioned on created_at, with a row-level immutability trigger).

Not in v0: intent classifier, OPA enforcement, Confluence/custom-API/web connectors, background enrichment workers, the Tempo/Loki/Prometheus/Grafana stack, a local-prod compose profile, and real auth.

Host ports below are the defaults; all of them can be moved in .env. None of these HTTP APIs authenticate callers except /mcp and the admin UI (see SECURITY.md).

Every graph write accepts an optional {"reason": "…"} and records a row in opencg.graph_audit_log.

The CLI inside the container (python -m opencg_ingestion.cli git <url> / re-extract) offers the same ingest plus re-extraction, without the URL restrictions.

Tools: opencg/retrieve_for_context and opencg/traverse_graph are live; opencg/search, opencg/get_entity and opencg/submit_feedback return not_implemented_in_mvp.

Browser code never calls the pipeline directly. It goes through same-origin Next route handlers: /api/proxy/search/vector, /api/proxy/entities/{id}, /api/proxy/ingestion/git/run and /api/proxy/graph/**. Also served: /api/healthz (pipeline-independent liveness) and /api/auth/login and /api/auth/logout (the access gate).

The init scripts run once, when the Postgres volume is first created, in file order: 010_extensions, 020_catalog, 030_audit, 040_graph, 050_graph_audit, 090_grants. They start at 010_ so Apache AGE's bundled 00-create-extension-age.sql runs first. There is no migration framework yet. A schema change means a new numbered file, and existing volumes need make down-v or a hand-applied script.

Extensions: age, pg_trgm, unaccent, uuid-ossp. An AGE graph named opencg holds a Concept vertex label and one edge label per vocabulary name. All tables live in the opencg schema, and every grant goes to the configured application role (OPENCG__POSTGRES__USER).

opencg.entity: the catalogue. One row per document (parent) and per chunk.

Indexes: (tenant, classification), (tenant, freshness_state), trigram GIN on body and title, and full-text GIN on title || body (the lexical retrieval leg).

opencg.audit_log: one row per retrieval. It is range-partitioned on created_at; today only the audit_log_default partition exists, and weekly partitions are planned. The primary key is (id, created_at). Columns: correlation_id, tenant, principal, roles text[], tool, query, intent_plan, retriever_versions, model_versions (jsonb), vector_collection, vector_snapshot_id, candidate_ids uuid[], candidate_decisions jsonb, final_entity_ids uuid[], final_context_hash, tokens_in, tokens_out, latency_ms, outcome (ok or error), error_code, legal_hold boolean. The trigger audit_log_no_update rejects every DELETE, and every UPDATE except setting legal_hold, which can never be cleared once set. Indexes: correlation_id; (principal, created_at desc); (tenant, created_at desc).

opencg.relationship_vocab: relationship types. Columns: name PK (^[a-z][a-z0-9_]*$), description, inverse, directed, deprecated_at, created_at. Ten are seeded:

• semantic: depends_on, defined_in, supersedes, mentions, related_to (undirected), causes, derived_from;
• code: calls, imports, uses.

opencg.concept: graph nodes. Columns: concept_id PK, tenant, name, dedupe_key (unique per tenant), description, aliases text[], state, confidence, extractor_version, symbol_id, symbol_kind, source_entity_id, created_at, updated_at, tombstoned_at. state is candidate, confirmed or tombstoned. Indexes: (tenant, state); (tenant, symbol_id) where set.

opencg.relationship_edge: graph edges. Columns: edge_id PK, tenant, type (→ relationship_vocab.name), from_concept_id, to_concept_id (→ concept), state (the same three values), confidence, extractor_version, created_at, updated_at, tombstoned_at. (tenant, from, type, to) is unique. Indexes: (state, confidence desc), from_concept_id, to_concept_id.

opencg.relationship_evidence: which chunks support an edge. Columns: evidence_id bigserial PK, edge_id (→ relationship_edge, on delete cascade), entity_id (→ entity), span, extractor_version, confidence, created_at. Indexes: edge_id, entity_id.

opencg.graph_audit_log: every graph state change. Columns: id, created_at, actor, tenant, target_kind (concept, edge or vocab), target_id, from_state, to_state, reason, before and after (jsonb). The trigger graph_audit_log_no_modify rejects every UPDATE and DELETE. Indexes: (target_id, created_at desc), (actor, created_at desc), (tenant, created_at desc).

One collection per source, named <collection_prefix><source> (default default__git), with vector_size 768 and cosine distance (opencg.yaml). Point IDs are chunk entity_ids. The payload holds tenant, entity_id, parent_entity_id, source, source_uri (the parent document's URI), title, text, classification, chunk_index, and symbol_id for code chunks. Postgres remains the source of truth: tombstones are applied at query time, not written to Qdrant.

Provisioned for caching, unused in v0.

• Retrieval. The dense (Qdrant) and lexical (Postgres full-text) legs run in parallel and are fused by reciprocal rank fusion. Assembly keeps candidates whose classification the caller's roles allow (admin and reader: internal, public; public: public; no matching role: public only), within the token budget. Every retrieval writes one audit row.
• Tombstones are soft and idempotent; the original timestamp is kept. A tombstoned entity is excluded from both retrieval legs and from vector search, while its Qdrant point stays. Re-ingesting the source clears the tombstone. Tombstoning does not cascade to chunks.
• Ingestion is idempotent. Stable uuid5 IDs mean re-ingesting unchanged content updates rows in place instead of duplicating them. Code files become one chunk per AST symbol (via graphifyy); text files are split into paragraph chunks.
• Graph states. graphifyy code relationships are written confirmed. Relationships extracted from text are candidate until an operator promotes them, and those below OPENCG__PROVIDERS__EXTRACTION__MIN_CONFIDENCE (0.6) or with an unknown type are dropped. Extraction is best-effort: failures are logged and counted and never fail an ingest.
• Audit logs are append-only. Database triggers enforce it; corrections are new rows.
• Token accounting. Every model call records its tokens on the OTel span and in opencg_tokens_total.
• "Run now" URL guard. URLs must be https, ssh, git or user@host:path, and must resolve to public addresses unless listed in OPENCG__INGESTION__GIT__ALLOWED_HOSTS. They may not embed a password, start with -, or use ext:: or file://. Local paths are allowed only under OPENCG__INGESTION__GIT__LOCAL_ROOTS.

CI runs static and dependency-review on every pull request, and the full-stack stack job (smoke plus e2e) on pull requests into main, nightly, and on demand. CONTRIBUTING.md explains each tier and how the e2e suite isolates its data.

• Intermittent hydration error. About 1 in 100 production page loads logs React error #418, most often on /entities. The page recovers on its own. The e2e suite retries once and annotates the report (#2).
• Ingestion metrics aren't exposed. The ingestion service counts tokens for its embedding and extraction calls, but serves no /metrics endpoint, so those counters can't be scraped. Its OTel spans carry the same token attributes.
• Audit partitioning is planned, not active. Every audit row lands in the default partition until weekly partitions are pre-created.
• No schema migrations. Changing an init script needs a fresh volume (make down-v).
• Run history is in memory. The ingestion run list clears when the service restarts.
• Moderate and low dependency advisories remain, each needing a major-version upgrade. They are listed with their exposure in SECURITY.md.
• v0 has no real authentication. The known stopgaps are listed in SECURITY.md.

Every feature, deviation, or course-correction goes through the OpenSpec change workflow before code is written. New work starts with openspec new change <name>, then proposal → design + specs → tasks → strict validation, then implementation phase by phase. Inline annotations in another change or undocumented pivots are not acceptable.

openspec list                                  # changes in flight
openspec list --specs                          # the archived spec baseline
openspec show add-foundation                   # the full v0 vision
openspec validate add-foundation --strict      # re-validate

Code-side ingestion uses graphifyy (MIT) for AST-level file discovery and symbol extraction across 28 tree-sitter languages. See openspec/changes/archive/2026-06-12-adopt-graphifyy/ for the integration contract.

MIT, copyright Electric Mind. Third-party components and their licences are listed in THIRD_PARTY_NOTICES.md. See CONTRIBUTING.md to get involved and SECURITY.md to report a vulnerability.