Zep gives you a temporal knowledge graph with explicit fact-validity windows. HeurChain gives you sub-50ms hybrid retrieval with no Neo4j. Both are credible choices — the right one depends on your domain. Here's how they actually differ.
Dataset: LongMemEval-S (ICLR 2025), 500 questions across 6 reasoning categories.
HeurChain numbers: measured on the heurchain-benchmarks harness (sharded_bench.py and multitenant_bench.py) against the broker in the main repo. Both are public; you can rerun the whole thing.
Zep numbers: from arXiv 2501.13956 (Zep / Graphiti) where available; left blank otherwise. We do not fabricate competitor numbers.
What we measured: retrieval R@k, MRR, NDCG@10, p50/p95 latency, and end-to-end QA accuracy with three independent judge models.
Cross-judge QA validation (May 2026): We ran the same retrieved facts through three judges from independent model families — full results published here. Mean QA accuracy (6 categories × 30 tasks): Local 14B 32.8%, DeepSeek V3.1 671B 31.7%, Kimi K2.6 28.3%. The two frontier judges agreed with each other on 87.8% of per-question verdicts, validating each as an independent judge. The local-14B mean was confirmed directionally correct within 4.5 pp of frontier judges — no inflation at the headline level.
What the per-category swings showed: the cross-judge run exposed a v2 fact-extraction quality bottleneck (specific entity-action assignments stripped to meta-summaries) on multi-session, knowledge-update, and temporal-reasoning categories. Where extraction preserves the answer-bearing detail, all three judges converge. Where it doesn't, the local 14B "won" by confabulating answers the local 14B judge then accepted — frontier judges honestly refused. Smoking-gun example in the writeup.
What we still owe: a v3 extraction prompt that preserves entity-action-value triples, and a closed-weight frontier judge run (Claude Sonnet 4.6 via Anthropic API) for additional independent validation. Both queued.
Bias disclosure: this is our internal harness, written by us. Of course it favors what we built well. The cross-judge run is the way we expose that bias and report it honestly. If you're evaluating both, the most reliable move is to run them on your data.
Zep's published evaluations focus on task accuracy with an LLM judge, not retrieval R@k. We don't have Zep R@k numbers to compare — fabricating them would be exactly the kind of "trust me bro" benchmarking we're trying to avoid.
Where this comparison gets muddy: Zep publishes task-completion accuracy (DMR / LongMemEval-style with an LLM judge), not retrieval R@k. Mixing the two would mislead. On retrieval-only metrics our numbers are what we measured; on Zep's published axes, Graphiti is genuinely strong at temporal reasoning. They're solving overlapping but different problems.
| Metric | HeurChain (dense) | HeurChain (hybrid α=0.9) | Zep (Graphiti) |
|---|---|---|---|
| R@1 | 0.543 | 0.542 | — |
| R@5 | 0.939 | 0.933 | — |
| R@10 | 0.972 | 0.978 | — |
| MRR | 0.911 | 0.913 | — |
| NDCG@10 | 0.911 | 0.914 | — |
Latencies from different harnesses on different deployment topologies — not strictly apples-to-apples. The multi-tenant Docker number is the closest analog to what a SaaS would actually serve. The in-process number shows what the algorithm itself is capable of with the network removed.
| System / configuration | P95 latency | Source | What it actually measures |
|---|---|---|---|
| HeurChain — multi-tenant load (Docker, 10 tenants concurrent) | 20.5 ms | This benchmark | Closest to production SaaS scenario |
| Zep / Graphiti | ~300 ms | arXiv 2501.13956 | Temporal KG traversal in critical path |
| HeurChain — dense, in-process | 35 ms | This benchmark | Algorithm-only ceiling; no network |
| HeurChain — BM25 only | 4.6 ms | This benchmark | Keyword-only path; useful for hot queries |
| Mem0 (reference) | 200 ms | Mem0 paper Table 1 | Search latency; stack-specific |
| LangMem (reference) | 59,820 ms | Mem0 paper Table 1 | Vector scan; broken at LongMemEval scale |
| HeurChain | Zep | |
|---|---|---|
| Retrieval method | BM25 + dense (bge-m3) + RRF (tunable α) | Temporal knowledge graph (Graphiti) — episodic + semantic node search |
| Storage backend | Redis (vectors + BM25) + SQLite (metadata) | Neo4j (graph) + Postgres + embedding store |
| Temporal awareness | Sequence-tagged facts (on roadmap) | First-class temporal facts with validity periods (Graphiti's whole point) |
| Multi-tenant model | Per-tenant namespace + agent_id sub-isolation; published zero-leak verification | Per-graph isolation; multi-tenant via Neo4j namespace setup |
| Self-hosted option | Single Go binary + Redis + SQLite | Docker Compose: Neo4j + Postgres + Zep services |
| API surface | REST + MCP SSE — auto-discovered by Claude Code, ChatGPT Apps | Python / TypeScript SDK + REST |
We're not going to pretend HeurChain wins on every dimension. These are real cases where Zep is the better fit:
Most readers should pick on architecture fit, not price. Zep Cloud has a free tier with limits and Hobby/Pro paid tiers; Graphiti is Apache-2.0 open source. HeurChain self-host is MIT-licensed. The numbers below exist so you can see them, not because we think they should drive your decision.
| If you... | HeurChain | Zep |
|---|---|---|
| Hobby / kicking the tires | Free self-host (MIT) | Free tier (Zep Cloud) or self-host Graphiti (Apache 2.0) |
| Solo developer, managed | $5/mo (Solo) | Zep Cloud Hobby / Pro tiers |
| Team, shared workspace | $49.99/mo (Workgroup) | Zep Cloud team pricing |
| Enterprise — SOC2, SAML | Custom | Custom |
Both have free options — Zep Cloud has a free tier, Graphiti is Apache-2.0 open source. If you're already running Neo4j in production, Zep self-host is essentially incremental. If you're not, the operational footprint matters.
python3 sharded_bench.py for the single-tenant baseline; python3 multitenant_bench.py --mode load --max-tenants 10 for the Docker multi-tenant number.Or self-host the same binary for free. If Zep fits your use case better, use Zep — we'd rather you pick the right tool.