A context graph is not a better RAG pipeline

The standard enterprise AI stack answers questions by fetching paragraphs. Chunk the documents, embed them, pull the twenty passages most similar to the question, hand them to a model. That architecture answers the questions an enterprise already knew how to answer. The questions worth the budget are relationship questions, and a retrieval pipeline cannot see a relationship it never stored.

This is the comparison buyers are now running under the label "context graph vs RAG," and it gets framed as a tooling decision. It is an architecture decision, and the deciding variable is the shape of the questions the business needs answered.

Two architectures, stated fairly

A retrieval-augmented generation pipeline does one job well. Documents are split into chunks, each chunk becomes a vector, and at question time the system retrieves the chunks whose vectors sit closest to the question's vector. The model writes an answer grounded in what came back. When the answer to a question is written down somewhere in one corpus, this works. A support desk answering from product documentation, a policy lookup against an underwriting manual, a paralegal searching case files: retrieval is the right architecture for all of them, and it ships in weeks.

A context graph stores something different. Entities, the relationships between them, the decisions made about them, the outcomes that followed, and the provenance of every record, connected across the systems where they originated. Answering a question means walking paths through those connections rather than matching text against text. The output of a traversal is a chain: this candidate, scored this way, on these signals, matched against this cohort, whose production history says this. What is a context graph covers the definition in full. The short version is that the graph keeps what retrieval throws away: the connections, the time dimension, and the evidence.

Neither substitutes for the other, and a buyer who frames this as better-or-worse will buy the wrong thing.

The question decides the architecture

Put a question to each architecture and watch what it needs.

"What does our underwriting manual say about coastal flood exposure?" is lookup-shaped. The answer exists as written text in one corpus. Retrieval wins on cost and speed to deploy, and a graph adds nothing but overhead.

"Which of these nine thousand applicants resembles the producers who survived four years?" is path-shaped. No document contains the answer. It lives across the ATS that holds the application, the assessment platform that scored the personality profile, the HRIS that recorded every promotion and exit, and the CRM transcripts that show how the strongest producers actually run a call. The answer is a set of connections between records that share no vocabulary. There is no paragraph to retrieve.

Microsoft Research hit this wall from inside retrieval research. Their GraphRAG work, with the method on arXiv, exists because baseline retrieval fails on questions that require connecting scattered pieces of information into a whole, so they built a knowledge graph over the corpus first and let the model traverse it. When the researchers who advanced retrieval respond to its limits by building a graph, the comparison has already been run once by the people with the least incentive to run it.

Anthropic's context-engineering guidance frames the underlying job precisely: find the "smallest possible set of high-signal tokens" for the model. Karpathy's frame says the same thing shorter: the model is the CPU, the context window is the RAM, and what fills the RAM decides the output. A retrieval pipeline fills the RAM with whatever sounds like the question. A context graph fills it with what is connected to the question. On lookup-shaped questions those are the same tokens. On path-shaped questions they are not even close.

Where the pipeline breaks

Three failure modes separate the architectures in production, and none of them is a bug: each is retrieval working as designed, outside its design.

Cross-system joins do not embed. Vector similarity connects text that shares meaning. A producer's ramp curve in the HRIS and their call pattern in the CRM share no words, so no embedding distance will ever place them near each other. The relationship between them is real, valuable, and invisible to similarity search, because it exists between records rather than inside any of them.

Provenance evaporates at chunk time. A retrieved chunk is a paragraph with its lineage stripped. Which system produced it, which version of the record it reflects, who touched it, what decision it fed: gone at ingestion. For a demo, that costs nothing. For a decision an auditor will examine, "these twenty chunks were in the window" is what the audit log can say, and an auditor who asks why the model recommended rejecting a candidate deserves better than a similarity score. A graph traversal, by contrast, is its own audit trail. Every hop is a recorded relationship with a source attached, which is what makes a Decision Trace queryable after the fact.

Time flattens. An embedding is a snapshot. "Who is our strongest producer" and "who was our strongest producer the quarter before she was promoted" collapse into nearly identical vectors, and the index has no native way to answer as-of. A graph carries validity windows on its edges, so the second question is a constrained traversal rather than a hope.

A team that needs none of this should not pay for a graph. Retrieval remains the correct architecture for the corpus it was designed for, and an honest vendor says so.

What the fusion is worth

The gap between the architectures has a measurement. At a Fortune 500 insurance carrier, across four years of production data and 10,765 agents, keyword screening alone scored an AUC of 0.558, barely above a coin toss. A personality assessment alone reached 0.647. The same signals fused across systems into one model, application data joined to assessment joined to behavioral history, reached 0.735. The lift never came from a better filter on any single source. It came from the connections between sources, which is the exact thing a per-corpus retrieval index cannot represent.

The single-source numbers are the empirical case against stopping at retrieval. That same study parsed 8,181 unique skills from four years of applicant data and found 3,597 that could be tested against post-hire production. After statistical correction, zero predicted the production milestone. The signal an enterprise pays for emerges only when systems are read together.

That fused model moderates rather than decides. It ranks candidates above a calibrated threshold for the same structured evaluation everyone gets, and a human makes the call, with the reasoning logged. The graph is what makes that reasoning inspectable at all.

Exit and security implications

The two architectures also age differently, and procurement should price that in.

A vector index is cheap to leave. Re-embedding a corpus with a new provider costs compute and a weekend. That low exit cost is genuine and counts in retrieval's favor. It is also a signal: the index accumulates little worth keeping, because it is a projection of documents that still exist elsewhere.

A context graph accumulates the record of decisions and outcomes, which is the asset that compounds. That raises the stakes on where it lives. A graph of performance histories, compensation-adjacent records, and call transcripts assembled on a vendor's multi-tenant infrastructure is a concentration of exactly the records a data-sensitive enterprise is least free to move. The Nodes answer is architectural: the graph is assembled and stays inside the customer's VPC, single-tenant, no data egress, and the model weights trained against it are customer-owned. If the relationship ends, the graph and the intelligence built on it remain in the customer's cloud. The context layer is the moat makes the strategic version of this argument; the security version is shorter. The most connected copy of your operational history should not be the copy you control least.

What to ask a vendor

Five questions separate a context graph from a retrieval pipeline wearing the label.

1. Ask a cross-system question in the demo and inspect what comes back. Paragraphs mean retrieval. A chain of linked records with sources attached means a graph.
2. Ask where the graph is assembled and what leaves the boundary. "We handle that" is an answer about their convenience.
3. Ask an as-of question. "Show me this team as it stood last March" is trivial on time-aware edges and nearly impossible on a static index.
4. Ask how traversal respects source-system permissions. A graph that flattens access control is a breach with good UX.
5. Ask what you keep at exit: the index, the graph, or the weights. The answer prices the relationship.

The first question

The architecture question is downstream of a simpler one: what shape are the questions your teams cannot currently answer? If they live inside one corpus, buy retrieval, spend the difference, and ship this quarter. If they cut across the ten to fifteen systems where your operation lives, no amount of retrieval tuning will connect records that were never connected. Retrieval fetches what is written. The graph knows what is related.

Sources

• GraphRAG: Unlocking LLM discovery on narrative private data
• From Local to Global: A GraphRAG Approach to Query-Focused Summarization
• Effective context engineering for AI agents
• Decision Traces: What Multi-System Data Fusion Reveals About Institutional Knowledge in Enterprise Hiring

---

Saad Bin Shafiq is the founder of Nodes, serving data-sensitive enterprises. Methodology: Decision Traces.