Give AI agents a programmer's brain — not text retrieval, but structural understanding.
SMP is a codebase intelligence server that models source code as a live, multi-dimensional knowledge graph. While traditional RAG treats code as flat text — leading to context overflow, stale hallucinations, and broken architectural awareness — SMP builds a structural model that AI agents can navigate, reason over, and safely mutate, even in codebases exceeding 100,000 lines.
- Why SMP
- Key Features
- Architecture Overview
- How It Works
- 1. Parser — AST Extraction
- 2. Graph Builder — Structural Analysis
- 3. Linker — Namespaced Cross-File Resolution
- 4. Call Resolution — Ingest + Placeholder Linking
- 5. Enricher — Static Metadata
- 6. Community Detection — Architectural Clustering
- 7. Locate — Keyword Search + Structural Expansion
- 8. Agent Safety Layer
- 9. Sandbox Runtime
- Quickstart
- Protocol Reference
- Agent Integration
- MCP Integration
- Technology Stack
- Project Structure
- Contributing
Standard RAG pipelines fail at code for three core reasons:
| Problem | What breaks | SMP's answer |
|---|---|---|
| Context overflow | 100k-line repos exceed any LLM window | Keyword search returns top-k, not the full graph |
| No structural awareness | Functions renamed, moved, or deleted invisibly | Live graph updated on every file change via watcher or git hook |
| Hallucinated dependencies | Flat-text models guess call chains | Same-file + global placeholder linking |
SMP replaces guessing with a graph where every node is a real code entity (function, class, file, interface) and every edge is a structural relationship (CALLS, IMPORTS, INHERITS, TESTS). Agents query the structure, not the text.
AI-First Architecture — Purpose-built to prevent agents from breaking on large codebases. Every response includes a pre-computed structural summary so agents read metadata first and drill into raw data only when needed.
MCP Native — Fully supports the Model Context Protocol, making SMP a plug-in memory layer for any MCP-compatible AI IDE or agent framework.
Keyword Search + Structural Traversal — smp/search and smp/locate score keywords against names, docstrings,
descriptions, tags, IDs, and file paths, then expand over the call/import graph. No vector seeding, no BM25.
Call Linking — Same-file calls are resolved at ingest; remaining cross-file calls are linked by global
name matching (resolve_placeholders). This over-approximates across files and languages — safe for impact
analysis, though same-named functions may link together.
Community Detection — Connected-component clustering over the call/import graph. Agents can query module boundaries and coupling weights between communities.
Blast Radius Analysis — Quantify the exact set of nodes affected by a change before a single line is edited. Impact analysis runs on the graph in milliseconds.
Merkle-Indexed Sync — SHA-256 Merkle tree over all file nodes. Incremental sync is O(log n) — only diverging subtrees are re-indexed. Snapshots are cryptographically signed for secure distribution to new agent instances.
Agent Safety Layer — Sessions with MVCC conflict detection, guard checks, dry-run impact preview, checkpoints, audit log, and per-node locking. Agents cannot accidentally overwrite concurrent work.
Sandbox Runtime — Process-level isolation: commands run as child processes inside a private working directory
with an optional command allowlist (SMP_SANDBOX_ALLOWED_COMMANDS). No containers, no eBPF trace capture.
Bring-Your-Own Embeddings — Ingest does NOT create embeddings and SMP ships no embedding model. Vectors live
in the FAISS-backed mmap vector store (.smpv) and are written only via the smp/vector/upsert API; keyword
search works without any vectors.
┌─────────────────────────────────────────────────────────────────┐
│ CODEBASE (Files + Git) │
└──────────────────────────┬──────────────────────────────────────┘
│ Updates (Watch / Agent Push / commit_sha)
▼
┌─────────────────────────────────────────────────────────────────┐
│ MEMORY SERVER (SMP Core) │
│ ┌─────────────┐ ┌──────────────┐ ┌─────────────┐ │
│ │ PARSER │──▶│ GRAPH BUILDER│──▶│ ENRICHER │ │
│ │ (Tree-sitter│ │ + LINKER │ │ (Static │ │
│ │ AST) │ │(Static) │ │ Metadata) │ │
│ └─────────────┘ └──────────────┘ └──────┬──────┘ │
│ │ │
│ ┌───────────────────────────────────────────▼──────────────┐ │
│ │ MEMORY STORE │ │
│ │ ┌──────────────────────────────────────────────┐ │ │
│ │ │ GRAPH STORE (MMapGraphStore, .smpg) │ │ │
│ │ │ Structure · CALLS · Sessions · Audit │ │ │
│ │ │ Keyword index (names/docs/tags/paths) │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ │ ┌──────────────────────────────────────────────┐ │ │
│ │ │ VECTOR STORE (FAISS mmap, .smpv) │ │ │
│ │ │ BYO embeddings via smp/vector/* API only │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ │ ┌──────────────────────────────────────────────┐ │ │
│ │ │ MERKLE INDEX │ │ │
│ │ │ SHA-256 per file · Package subtree hashes │ │ │
│ │ │ Root hash = full codebase state │ │ │
│ │ └──────────────────────────────────────────────┘ │ │
│ └──────────────────────────────┬───────────────────────────┘ │
└─────────────────────────────────┼───────────────────────────────┘
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌──────────────────────┐ ┌───────────────┐
│ QUERY ENGINE │ │ SANDBOX RUNTIME │ │ SWARM LAYER │
│ Navigator │ │ Child processes in │ │ Peer Review │
│ Reasoner │ │ private work dirs │ │ PR Handoff │
│ Keyword locate │ │ Command allowlist │ └───────┬───────┘
│ Telemetry │ │ Process isolation │ │
└────────┬────────┘ └──────────┬───────────┘ │
└──────────────┬────────┘ ────────┘
│ SMP Protocol (Dispatcher)
▼
┌─────────────────────────────────────────────┐
│ AGENT LAYER │
│ Agent A Agent B Agent C │
│ (Coder) (Reviewer) (Architect) │
└─────────────────────────────────────────────┘
Technology: Tree-sitter (multi-language, fast, incremental)
Tree-sitter parses every source file into a typed Abstract Syntax Tree. The parser extracts functions, classes, variables, interfaces, imports, and exports — producing a structured document for the Graph Builder to consume.
Supported languages (14): Python; JavaScript (.js/.jsx/.mjs/.cjs); TypeScript (.ts/.mts/.cts) and TSX;
Java; C; C++; C#; Go; Rust; PHP; Ruby; Swift; Kotlin; MATLAB (see _LANGUAGE_SPECS in
smp/store/graph/parser.py and DEFAULT_EXTENSIONS in smp/cli.py).
Extracted per file:
{
"file_path": "src/auth/login.ts",
"language": "typescript",
"nodes": [
{
"id": "func_authenticate_user",
"type": "function_declaration",
"name": "authenticateUser",
"start_line": 15,
"end_line": 42,
"signature": "authenticateUser(email: string, password: string): Promise<Token>",
"docstring": "Validates user credentials and returns JWT...",
"modifiers": ["async", "export"]
},
{
"id": "class_AuthService",
"type": "class_declaration",
"name": "AuthService",
"methods": ["login", "logout", "refresh"],
"properties": ["tokenExpiry", "secretKey"]
}
],
"imports": [
{"from": "./utils/crypto", "items": ["hashPassword", "compareHash"]},
{"from": "../db/user", "items": ["UserModel"]}
],
"exports": ["authenticateUser", "AuthService"]
}The Graph Builder transforms AST output into a property graph stored in the memory-mapped journal graph store
(.smpg files, smp/store/graph/). The store is self-contained — no external database. Every code entity
becomes a node; every structural dependency becomes a typed, directed edge.
Node types:
| Node | Represents |
|---|---|
Repository |
Root node for the entire codebase |
Package |
Directory or module |
File |
Source file |
Class |
Class definition |
Function |
Function or method |
Variable |
Variable or constant |
Interface |
Type definition or interface |
Test |
Test file or test function |
Config |
Configuration file |
Community |
Connected-component cluster |
Relationship types:
| Relationship | Meaning |
|---|---|
CONTAINS |
Parent-child (Package → File) |
IMPORTS |
File imports File / Module |
DEFINES |
File defines Class / Function |
CALLS |
Function calls Function (namespaced) |
INHERITS |
Class inherits Class |
IMPLEMENTS |
Class implements Interface |
DEPENDS_ON |
General dependency |
TESTS |
Test covers Function / Class |
USES |
Function uses Variable / Type |
REFERENCES |
Variable references Variable |
MEMBER_OF |
Node belongs to Community |
BRIDGES |
Community connects to Community |
The Linker runs at ingest and again afterwards. Calls whose target is defined in the same file are resolved
immediately; every other call is recorded as an edge to a ::name:: placeholder. resolve_placeholders then
links each placeholder to every node with a matching structural name — across files AND languages.
This deliberately over-approximates: the safe direction for impact analysis, since no caller is missed. The trade-off is that same-named functions may link together:
File A calls: save()
File B has: save() (src/db/user.py)
File C has: save() (src/cache/session.py)
→ placeholder ::save:: links to BOTH definitions
Placeholders are kept so later ingests can link newly added files, and edge dedup keeps re-runs idempotent.
SMP's call graph is static only. There is no runtime linker: no eBPF collector, no kernel-level execution
tracing, and no sandbox trace ingestion. Dynamic dispatch (dependency injection, event buses, metaprogramming)
is visible only where a same-named definition exists statically — resolve_placeholders will link it, along
with any other same-named definitions.
The Enricher attaches human-readable metadata to structural nodes using only what already exists in the code: docstrings, inline comments, decorators, and type annotations. No LLM. Pure static extraction.
Ingest does NOT create embeddings and SMP ships no embedding model. The FAISS-backed mmap vector store (.smpv,
smp/vector/) accepts bring-your-own embeddings via the smp/vector/upsert API only. Keyword search
(smp/search, smp/locate) works without any vectors.
Enriched node schema (final):
{
"id": "func_authenticate_user",
"semantic": {
"status": "enriched",
"docstring": "Validates user credentials and returns a signed JWT.",
"inline_comments": [
{"line": 18, "text": "compare against bcrypt hash, not plaintext"}
],
"decorators": ["@requires_db", "@rate_limited"],
"annotations": {
"params": {"email": "string", "password": "string"},
"returns": "Promise<Token>",
"throws": ["AuthenticationError", "DatabaseError"]
},
"tags": ["auth", "jwt", "session"],
"source_hash": "a3f9c12d",
"enriched_at": "2025-02-15T10:30:00Z"
}
}Purpose: Automatically partition the codebase graph into structural clusters so agents can reason about domain boundaries and coupling between modules.
Method: Connected-component analysis over the undirected projection of the graph, optionally filtered by
relationship types (smp/community/detect). Results are cached server-side so smp/community/list,
smp/community/get, and smp/community/boundaries return consistent IDs. No Louvain, no PageRank, no LLM.
smp/locate is SMP's primary feature discovery endpoint. It scores every node against the query terms and
returns the top matches with what each matched on (name, docstring, tags):
smp/locate: all query terms in name → 100 · any term in name → 50 ·
all terms in docstring → 30 · any term → 15 · tag match → +10
smp/search: name +3 · node id +3 · file path +1 · docstring +2 · description +1
smp/search adds match (any/all), node_types / tags / scope filters, and top_k. Both are plain
scored keyword matching over the graph store — no BM25 index, no vector seeding, no PageRank.
SMP provides a full safety harness for agents operating in write mode:
Sessions — Every write operation must open a session declaring its scope and intent. Sessions and locks are persisted in the graph store, with exclusive locks for write sessions.
Guard Checks (smp/guard/check) — Pre-flight check before any write. Returns blocked, warning, or clear based on concurrent session conflicts, hot-node status (heat score > 90), lock status, and test coverage gaps.
Dry Run (smp/dryrun) — Proposes a change and receives a full impact preview: breaking vs. non-breaking verdict, list of affected callers, missing tests, and structural diff — before touching disk.
Checkpoints (smp/checkpoint) — Snapshot the current graph state for a set of files before writing. Enables rollback if a change produces unexpected results.
Audit Log — Every session, guard check, dry run, checkpoint, and write is recorded with timestamp and agent
ID. Queryable via smp/audit/log.
Every sandbox is an isolated execution environment:
- Child processes in a private working directory — process-level isolation, not containers
- Command allowlist — restrict runnable commands via
SMP_SANDBOX_ALLOWED_COMMANDS - Timeouts and stdin capture — bounded execution with stdout/stderr capture
Sandboxes are used for safe execution of agent-proposed code before committing.
Requirements: Python 3.11+ (see requires-python = ">=3.11" in pyproject.toml). No external database — the
graph lives in a self-contained .smpg file and vectors (optional, bring-your-own) in .smpv.
# 1. Clone and configure
git clone https://github.com/your-org/smp.git
cd smp
cp .env.example .env
# 2. Set up Python environment
python3.11 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
# 3. Start the server (writes to .smp/graph.smpg by default)
smp serve --port 8420 --graph-path .smp/graph.smpg
# 4. Ingest your project (structural parsing only — creates NO embeddings)
smp ingest /path/to/your/project --graph-path .smp/graph.smpg
# 5. Run a query
curl -X POST http://localhost:8420/rpc \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "id": 1, "method": "smp/locate", "params": {"query": "authentication logic"}}'Environment variables (.env):
SMP_GRAPH_PATH=.smp/graph.smpg
SMP_VECTOR_PATH=.smp/smp.smpv
SMP_HOST=0.0.0.0
SMP_PORT=8420
# Optional Redis for distributed rate limiting
SMP_REDIS_URL=SMP uses JSON-RPC 2.0 over HTTP (smp serve, POST /rpc). AI agents can also connect via MCP over stdio
(smp mcp). Every method follows the same envelope:
{
"jsonrpc": "2.0",
"method": "smp/<method>",
"params": { ... },
"id": 1
}{
"jsonrpc": "2.0",
"method": "smp/update",
"params": {
"file_path": "src/auth/login.ts",
"content": "...",
"change_type": "modified" // "modified" | "created" | "deleted"
},
"id": 1
}Response:
{
"result": {
"status": "success",
"nodes_added": 3,
"nodes_updated": 12,
"nodes_removed": 1,
"relationships_updated": 8
}
}{
"method": "smp/batch_update",
"params": {
"changes": [
{"file_path": "src/auth/login.ts", "content": "...", "change_type": "modified"},
{"file_path": "src/auth/middleware.ts", "content": "...", "change_type": "created"}
]
}
}Sends client root hash + per-file SHA-256 hashes. Server compares against its Merkle tree and returns exactly which files need to be pushed or pulled.
{
"method": "smp/sync",
"params": {
"client_root_hash": "e3b0c44298fc",
"file_hashes": {
"src/auth/login.ts": "a3f9c12d",
"src/utils/crypto.ts": "c3a1f004"
}
}
}{
"method": "smp/index/export",
"params": {
"scope": "full",
"signing_key_id": "key_prod_01"
}
}{
"method": "smp/index/import",
"params": {
"snapshot_id": "snap_4f8a2c",
"source_url": "smp://snapshots/snap_4f8a2c.tar.zst",
"expected_root_hash": "f7c2a19b3d84",
"verify_signature": true
}
}{
"method": "smp/navigate",
"params": {
"query": "authenticateUser",
"include_relationships": true
}
}{
"method": "smp/trace",
"params": {
"start": "func_authenticate_user",
"relationship": "CALLS",
"depth": 3,
"direction": "outgoing"
}
}{
"method": "smp/flow",
"params": {
"entry": "func_authenticate_user",
"direction": "out",
"depth": 4
}
}{
"method": "smp/diff",
"params": {
"from_sha": "abc1234",
"to_sha": "def5678",
"scope": "package:src/auth"
}
}{
"method": "smp/why",
"params": {
"from": "func_authenticate_user",
"to": "class_UserModel"
}
}Returns a pre-computed structural summary (role, blast radius, risk level, test coverage, heat score) plus raw graph data: imports, importers, defined symbols, structurally similar files, entry points, and data flow.
{
"method": "smp/context",
"params": {
"file_path": "src/auth/login.ts",
"scope": "edit" // "edit" | "create" | "debug" | "review"
}
}Summary fields in the response:
| Field | Description |
|---|---|
role |
Topology-derived: endpoint, service, core_utility, test, config, isolated, module |
blast_radius |
Number of files that import this file |
api_layer_callers |
Callers originating from the API layer |
avg_complexity |
Average cyclomatic complexity of defined functions |
max_complexity |
Highest complexity function in the file |
has_tests |
Whether test coverage exists |
is_hot_node |
True if heat score > 90 |
heat_score |
Frequency of recent access (0–100) |
risk_level |
high / medium / low — derived from blast_radius and complexity |
{
"method": "smp/impact",
"params": {
"entity": "func_authenticate_user",
"change_type": "signature_change" // "signature_change" | "delete" | "move"
}
}{
"method": "smp/locate",
"params": {
"query": "user registration flow",
"fields": ["name", "docstring", "tags"],
"top_k": 10
}
}Returns ranked matches with entity, file, matched_on, docstring, and tags. Scoring is plain keyword
matching (name hits outrank docstring hits; tag hits add on) — no vectors, no PageRank.
{
"method": "smp/community/detect",
"params": {
"relationship_types": ["CALLS", "IMPORTS"],
"resolutions": []
}
}{"method": "smp/community/list", "params": {"level": 1}}{
"method": "smp/community/get",
"params": {
"community_id": "comm_auth_core",
"node_types": ["Function", "Class"],
"include_bridges": true
}
}{
"method": "smp/community/boundaries",
"params": {"level": 0, "min_coupling": 0.05}
}Returns coupling weights and the specific bridge nodes responsible for cross-domain dependencies.
{"method": "smp/enrich", "params": {"node_id": "func_authenticate_user", "force": false}}Skips silently if source_hash is unchanged since last enrichment.
{"method": "smp/enrich/batch", "params": {"scope": "package:src/auth", "force": false}}{"method": "smp/enrich/stale", "params": {"scope": "full"}}Returns total_nodes, has_docstring, has_annotations, has_tags, no_metadata, stale, and coverage_pct.
Used for no_metadata nodes that have nothing extractable from the AST.
{
"method": "smp/annotate",
"params": {
"node_id": "func_xT9_handler",
"description": "Processes Stripe webhook payload and updates subscription status.",
"tags": ["billing", "webhook", "stripe"]
}
}{
"method": "smp/tag",
"params": {
"scope": "package:src/payments",
"tags": ["billing", "stripe", "pci-sensitive"],
"action": "add" // "add" | "remove" | "replace"
}
}Scored keyword matching over the graph store (name/id/path/docstring/description weights). No BM25 index.
{
"method": "smp/search",
"params": {
"query": "stripe webhook",
"match": "all",
"filter": {
"node_types": ["Function", "Class"],
"tags": ["billing"],
"scope": "package:src/payments"
},
"top_k": 5
}
}{
"method": "smp/session/open",
"params": {
"agent_id": "agent_coder_01",
"task": "Refactor authentication middleware",
"scope": ["src/auth/login.ts", "src/auth/middleware.ts"],
"mode": "write" // "read" | "write"
}
}{
"method": "smp/guard/check",
"params": {
"session_id": "sess_abc123",
"target": "src/auth/login.ts"
}
}Returns verdict: clear, warning, or blocked along with reasons and recommended actions.
{
"method": "smp/dryrun",
"params": {
"session_id": "sess_abc123",
"file_path": "src/auth/login.ts",
"proposed_content": "..."
}
}Returns verdict: safe or breaking, with the list of affected nodes, missing tests, and a structural diff.
{
"method": "smp/checkpoint",
"params": {
"session_id": "sess_abc123",
"files": ["src/auth/login.ts"]
}
}{
"method": "smp/session/close",
"params": {"session_id": "sess_abc123", "status": "completed"}
}{
"method": "smp/audit/log",
"params": {
"agent_id": "agent_coder_01",
"since": "2025-02-15T00:00:00Z",
"event_types": ["session_open", "dryrun", "write"]
}
}{
"method": "smp/sandbox/spawn",
"params": {
"name": "auth-tests",
"template": null,
"files": {}
}
}{
"method": "smp/sandbox/execute",
"params": {
"sandbox_id": "box_99x",
"command": ["npm", "test", "--", "src/auth"],
"timeout": 120
}
}{"method": "smp/sandbox/kill", "params": {"execution_id": "box_99x"}}{
"method": "smp/handoff/review",
"params": {
"session_id": "sess_abc123",
"reviewer_agent": "agent_reviewer_01",
"notes": "Refactored token expiry handling."
}
}Returns a PR package containing: changed files, structural diff, and guard check history.
import asyncio
from smp.client import SMPClient
async def main():
async with SMPClient("http://localhost:8420") as client:
# Feature discovery
results = await client.locate("user registration flow")
# Impact analysis
impact = await client.assess_impact("src/auth/manager.py::authenticate")
print(f"Change affects {impact['total_affected_nodes']} nodes")
# Get editing context
context = await client.get_context("src/auth/login.ts", scope="edit")
print(f"Risk level: {context['summary']['risk_level']}")
print(f"Blast radius: {context['summary']['blast_radius']} files")
asyncio.run(main())import { SMPClient } from "@smp/client";
const client = new SMPClient("http://localhost:8420");
// Locate a feature
const results = await client.locate("payment webhook handler");
// Assess impact before editing
const impact = await client.impact("func_process_payment", "signature_change");
console.log(`Affects ${impact.total_affected_nodes} nodes`);This is the recommended pattern for any agent performing a write operation:
class CodingAgent:
def __init__(self, smp_client):
self.smp = smp_client
def edit_file(self, file_path: str, instruction: str, new_code: str):
# 1. Open a session — declare scope and intent upfront
session = self.smp.call("smp/session/open", {
"agent_id": self.agent_id,
"task": instruction,
"scope": [file_path],
"mode": "write"
})
# 2. Pre-flight guard check — abort immediately if blocked
guard = self.smp.call("smp/guard/check", {
"session_id": session["session_id"],
"target": file_path
})
if guard["verdict"] == "blocked":
raise AbortError(guard["reasons"])
# 3. Get full structural context — agents read summary first
context = self.smp.call("smp/context", {
"file_path": file_path,
"scope": "edit"
})
# 4. Dry run — preview impact before touching disk
dryrun = self.smp.call("smp/dryrun", {
"session_id": session["session_id"],
"file_path": file_path,
"proposed_content": new_code,
})
if dryrun["verdict"] == "breaking":
raise AbortError(dryrun["risks"])
# 5. Checkpoint → write → sync memory
self.smp.call("smp/checkpoint", {
"session_id": session["session_id"],
"files": [file_path]
})
write_to_disk(file_path, new_code)
self.smp.call("smp/update", {
"file_path": file_path,
"content": new_code,
"change_type": "modified"
})
# 6. Close session
self.smp.call("smp/session/close", {
"session_id": session["session_id"],
"status": "completed"
})SMP is a native MCP server over stdio. Add it to your agent's MCP configuration to expose SMP methods as tools:
{
"mcpServers": {
"smp": {
"command": "smp",
"args": ["mcp", "--graph-path", ".smp/graph.smpg"]
}
}
}Once connected, your MCP-compatible IDE or agent (Cursor, Claude Code, Windsurf, etc.) will have access to all smp/* methods as first-class tools, with full structural memory for every code change.
| Component | Technology | Rationale |
|---|---|---|
| AST Parsing | Tree-sitter (14 languages, see §1) | Functions, classes, interfaces, calls — no LLM |
| Graph Store | MMapGraphStore (.smpg journal) |
Self-contained; no external DB |
| Vector Store | FAISS-backed mmap (.smpv) |
Bring-your-own embeddings via smp/vector/*; ingest creates none |
| Merkle Index | SHA-256 (in-process) | O(log n) incremental sync, secure snapshot distribution |
| Community Detection | Connected components (in-process) | Topology-only, no LLM, reproducible |
| Sandbox Runtime | Child processes in private work dirs | Bounded execution with command allowlist |
| Backups | Gzipped tarballs (snapshot + manifest.json) |
Via smp backup / smp restore |
| Data Models | msgspec | Zero-copy, schema-validated structs |
| Protocol | JSON-RPC 2.0 over HTTP + MCP over stdio | Standard, simple, agent-compatible |
| Embeddings | None built in (BYO only) | No embedding model ships with SMP; ingest creates no vectors |
| Language | Python 3.11+ (requires-python = ">=3.11") |
Modern typing (X | Y), tomllib, tree-sitter |
smp/
├── cli.py # CLI: ingest | serve | mcp | backup | restore | compact | integrity
├── logging.py # Structured logging
├── core/
│ ├── models.py # GraphNode, GraphEdge, NodeType, EdgeType, method params (msgspec)
│ └── config.py # Settings from env (SMP_GRAPH_PATH, SMP_VECTOR_PATH, SMP_HOST/PORT)
├── engine/
│ ├── query.py # Keyword locate/search, navigate, trace, context, impact, flow
│ └── graph_builder.py # Graph ingestion and edge resolution
├── store/
│ ├── interfaces.py # Abstract store interfaces
│ └── graph/ # Memory-mapped journal graph store (.smpg)
│ ├── mmap_store.py # MMapGraphStore + resolve_placeholders
│ ├── parser.py # Tree-sitter structural parsing (14 languages)
│ ├── query.py # Graph queries
│ └── journal.py # Append-only journal
├── vector/ # FAISS-backed mmap vector store (.smpv, BYO embeddings only)
│ ├── mmap_vector.py
│ └── faiss_index.py
├── protocol/
│ ├── server.py # FastAPI app: JSON-RPC over HTTP (~50 smp/* methods)
│ ├── mcp.py # MCP server over stdio
│ └── handlers/ # query, memory, enrichment, community, session, vector, sandbox, ...
├── runtime/
│ └── sandbox.py # Child-process sandbox runtime
├── observability/
│ └── backup.py # backup/restore/compact (gzipped tarball + manifest.json)
└── ...
Handler pattern — each method group lives in its own handler module with plain async functions.
# protocol/handlers/query.py
async def search(params: dict[str, Any], ctx: dict[str, Any]) -> dict[str, Any]:
p = msgspec.convert(params, SearchParams)
engine = ctx["engine"]
return await engine.search(p.query, p.match, p.filter, p.top_k)
# protocol/handlers/community.py
async def community_detect(params: dict[str, Any], ctx: dict[str, Any]) -> dict[str, Any]:
return await detect(params.get("relationship_types"), ctx)| Component | Purpose |
|---|---|
| Parser | Extract AST from source (Tree-sitter, 14 languages) |
| Graph Builder | Create structural nodes and relationships |
| Linker | Same-file call resolution at ingest + global name-based placeholder linking |
| Enricher | Attach docstrings, annotations, tags (static only; no embeddings) |
| Graph Store | MMapGraphStore — memory-mapped journal (.smpg), keyword index, sessions, telemetry |
| Vector Store | FAISS-backed mmap (.smpv) — bring-your-own embeddings via smp/vector/* |
| Merkle Index | SHA-256 tree — O(log n) incremental sync + secure distribution |
| Query Engine | navigate, trace, context, impact, locate, search, flow, diff, why |
| SMP Protocol | JSON-RPC 2.0 over HTTP (smp serve) + MCP over stdio (smp mcp) |
| Agent Safety | Sessions, guard checks, dry runs, checkpoints, audit log |
| Telemetry | Hot node tracking, heat scores, automatic safety escalation |
| Community Detection | Connected components over the call/import graph + boundary queries |
| Sandbox Runtime | Child processes in private work dirs, command allowlist |
| Integrity Gate | Full on-disk integrity check (smp integrity) |
| Swarm Handoff | Peer review pass-off + structured PR with structural diff |
See CONTRIBUTING.md for setup instructions, coding standards, and how to add new protocol methods or language parsers.
- Architecture Guide — Deep dive into the Graph RAG pipeline and storage layer.
- API Reference — Full JSON-RPC 2.0 method specification with all parameters and response schemas.
- User Guide — Tutorials and advanced agent workflows.
- Contributing — How to extend SMP with new parsers, methods, and integrations.
SMP — giving AI agents the structural memory to master any codebase.