Vector Search Is a Strategy, Not an Architecture

Compile vs. retrieve - what LLM Wiki gets right, and what I’d keep from the graph.

Every RAG tutorial ends in the same place. Chunk the documents, embed the chunks, push them into a vector store, retrieve the top k by cosine similarity, stuff them into the prompt. It works. You demo it, everyone nods, you ship it.

Then someone asks a real question.

The query that broke my index

I built Cortex - a data-mesh memory engine for LLMs - partly because of questions shaped like this one. Run it through a pure vector index and you get back a perfectly reasonable set of chunks. The incident postmortem. A Slack thread with the word “outage” in it. The runbook page titled “Outage response.” Every one of them scores well. Every one of them is about the outage.

None of them is the answer.

Cosine similarity optimizes for aboutness. The question asked for causality and order. Those are not the same thing, and no amount of better embeddings closes the gap.

When vector-only retrieval failed, my instinct was to reach for the usual knobs: a better embedding model, smaller chunks with more overlap, a bigger k, a reranker on top. Each bought a little. None of them fixed it - because the question and the index were asking about different dimensions of the data.

Strip the tutorial framing away and vector search makes exactly one claim: this text is semantically near that text. It is genuinely good at that claim. It is silent on three others that real questions depend on.

• Cause. A postmortem mentioning a bad deploy and the incident it triggered are semantically similar to each other and to fifty other deploy-related documents. “Caused” is a directed edge between two specific events. An embedding flattens direction into proximity - you cannot recover an arrow from a distance.
• Order. “What happened, in what order” is a sequence. A top-k list is a set. You can retrieve all seven events of an incident and still hand the model a shuffled deck.
• Time. A config change from last Tuesday and an identical one from eighteen months ago embed to nearly the same point. To the index they are interchangeable. To the question they could not be more different.

So I built a better retriever

My answer was to stop treating similarity as the system. In Cortex, vector search is one of seven strategies. Seven run in parallel on every query - vector, hybrid BM25+dense, temporal, episodic, causal, graph, preference - and fuse into a single ranked bundle.

They matter because they fail differently. Vector search fails by returning things that are topically adjacent but useless. Graph traversal fails when an edge was never wired at ingestion. Temporal retrieval fails when the answer is genuinely old. Three uncorrelated failure modes, fused, land on the answer far more often than the best single retriever tuned for a year.

For the outage query, the causal strategy does not search at all. It walks:

MATCH (e:Event)-[:CAUSED_BY]->(cause:Event)
WHERE e.id = $incident_id
OPTIONAL MATCH (e)-[:FIXED_BY]->(fix:Event)
RETURN cause, fix

That returns the cause and the fix as nodes, with the edge connecting them. Not a chunk that happens to mention both. No embedding, no similarity threshold to tune. The relationship was either extracted at write time or it was not.

A lightweight classifier produces an intent distribution - for the outage query, roughly causal 46%, recall 27%, semantic 18%, preference 9% - which re-scales the fusion weights. It never gates a strategy.

The seven ranked lists merge through weighted reciprocal-rank fusion - deduplicated by id, normalized with the intent-scaled weights. RRF matters specifically because it consumes ranks, not scores. A FAISS inner-product score, a graph-hop distance, and a recency decay are not on comparable scales and never will be. Rank is the only currency all seven can honestly spend.

And every hit carries its own provenance:

{
  "id": "evt_8f2a",
  "kind": "event",
  "score": 0.91,
  "why": {
    "strategy": "causal",
    "path": "Incident:friday-outage -[CAUSED_BY]-> Event:deploy-4471",
    "hops": 1,
    "recency": 0.84,
    "weight": 1.15
  }
}

The run persists as a RetrievalTrace and the bundle returns a trace_id - a permalink to its own reasoning. When retrieval goes wrong, the question is never “why did the model say that.” It is “which strategy surfaced this, at what weight, through which path.” That has a lookup-able answer. Vector-only RAG cannot offer this - not because nobody implemented it, but because there is nothing to say. The explanation for a cosine hit is that the vectors were close, which is not a reason. It is a restatement of the mechanism.

I thought that was the whole argument. Retrieve better. Seven strategies beat one.

Then I read Karpathy’s gist and realised I had answered a smaller question than the one being asked.

The axis I was missing

In April 2026 Andrej Karpathy published a short gist describing what he calls an LLM Wiki. It is not a tool or a package - he calls it an idea file, meant to be pasted into your own agent. The pattern: instead of indexing raw documents for retrieval, an LLM agent incrementally reads each source and integrates it into a persistent, interlinked set of markdown pages. Entity pages, concept pages, cross-links, contradictions, source citations. The wiki compounds. Every source you add makes it richer.

His critique of RAG is not that similarity is imprecise. It is that RAG never accumulates. The model rediscovers the same knowledge from scratch on every single question.

Around the same time, a team at Tencent published a paper - Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki (arXiv 2605.25480). Same name, independent lineage; it is not a formalisation of Karpathy’s gist, though the two are frequently conflated. Their argument is the same one in academic register: RAG organises knowledge as flat chunks retrieved by embedding similarity, which exposes a retrieval-as-lookup interface - the wrong shape for an agent that needs to search, read, traverse, and decide when it has seen enough.

Their motivating example is, almost exactly, mine. A four-hop question: which of two films had the older director? A dense retriever happily returns both film pages and misses the director biographies holding the birth dates - because those pages are semantically distant from the question as asked. Aboutness again. The answer is two hops away from anything the query is about.

They compile documents into wiki pages with bidirectional links, expose search, read, and link-following as tool calls, and add an Error Book that accumulates corrections across ingestion batches. On HotpotQA, MuSiQue and 2WikiMultiHopQA they report state-of-the-art, beating HippoRAG 2, LightRAG and GraphRAG by 2.0-8.1 F1 - with the gains widening as hop count rises.

The real axis was never seven strategies versus one. It is when you pay for synthesis: at read time, on every query, forever - or once, at write time.

That reframing stung, because measured against it, my seven-strategy pipeline is still a read-time system. Elaborate, fused, traced - and still re-deriving the shape of the answer on every request.

The uncomfortable part: I already agree

Here is what I noticed reading the gist. Cortex’s write path is already a compilation step. Ingestion is async and never blocks a request, and it does far more than embed:

Upload (POST /api/documents)
  -> Parse + Chunk
  -> Embed -> FAISS
  -> Summarize + Graph-wire      # link_related_chunks
  -> Extract entities / episodes # extract_document_entities
  -> Infer causal edges          # CAUSED_BY / FIXED_BY

Summarising at ingestion. Linking related chunks across documents. Extracting entities and wiring MENTIONS and DEPENDS_ON edges. Inferring causality between events. Every one of those is knowledge compiled once at write time so it does not have to be re-derived at read time. That is Karpathy’s thesis. I had been doing it for months while writing blog posts about retrieval.

So the disagreement is narrower than it first looks, and much more interesting. We agree synthesis belongs at write time. We disagree about what to compile into.

The wiki’s artifact is prose a human can read, audit, and fix by hand. That is a real advantage and I do not want to wave it away - a markdown page in git has a diff, a history, and a reviewer. My Neo4j edges have none of that. Nobody browses a graph over coffee and notices it is wrong.

But prose has a cost the wiki papers are quiet about. A wiki page is a lossy compilation. Once the agent has written “the deploy at 14:02 likely caused the outage,” the hedge, the timestamp and the causal claim are fused into a sentence - and the next question has to re-parse them back out of English. A CAUSED_BY edge with a confidence property does not need re-parsing. It can be traversed, weighted, and counted. It can also be wrong in a way that shows up in a query rather than hiding in a paragraph.

And the failure modes differ in a way that matters operationally. When my graph is wrong, an edge is missing and the causal strategy silently returns nothing - loud, debuggable, and covered by the other six strategies. When a wiki is wrong, a page confidently asserts something false, and every future question that reads that page inherits the error. Compilation errors compound exactly as fast as compiled knowledge does. The Error Book exists precisely because the Tencent team hit this.

A layer, not a product

The honest conclusion is that these are not competitors. A wiki page is a compiled artifact; a graph is a compiled index. Neither is a system. Both are layers, and the useful thing about a layer is that it composes - which is why this pattern is worth your attention whether or not you have any infrastructure at all.

It has two scales, and they are further apart than almost anything else in this space.

On its own: a folder, an agent, and git

Karpathy’s version needs no vector database, no graph database, no ingestion pipeline, no service. The floor of this pattern is a directory of sources you never edit, a directory of markdown the agent maintains, and git for history. That is the entire dependency list. You can start this afternoon with an agent and a text editor.

The gist points it at three things: personal knowledge - goals, health, journal entries; research - papers and articles feeding an evolving thesis; and reading - chapter-by-chapter synthesis with pages for characters and themes. In each case the agent reads a new source, folds it into the existing pages, adds cross-links, and flags where the new material contradicts what is already written. You curate sources and ask questions; the LLM does the bookkeeping.

Almost every piece of retrieval advice has a floor of “first, stand up a vector database.” This one’s floor is a folder. That is not a small thing.

Be clear-eyed about why it stays simple, though. At that scale there is no retrieval problem to solve - you have forty pages, not four million, so “search” is reading the index and opening the right file. The pattern’s elegance is partly a function of its size. That is not a criticism; it is the regime it was designed for, and most people are in that regime. Know which one you are in before you copy the architecture.

What survives the scale change is the idea, not the implementation: read the source once, write down what you learned, link it to what you already knew. Everything below is that same sentence with more machinery around it.

As a layer: the Cortex case

Scale up and the pattern needs a home. Cortex already has the write path, the store, and the fusion machinery to hold one, so here is what it looks like as a component rather than a product - and the shape generalises to any system with an ingestion pipeline and more than one retriever.

• A :WikiPage node type. One page per entity, compiled by an agent from every chunk that MENTIONS it, stored as a node with the markdown as a property and its sources as edges. The graph already knows which chunks mention an entity - that part exists, and it would be the compile input, for free.
• A PageWriter, mirroring EpisodeWriter. Episodes already go through a single EpisodeWriter that mirrors to the graph and embeds on write. Pages would get the same treatment: one writer, one place where compilation happens, one place to fix when it is wrong.
• An eighth strategy: wiki. It would retrieve the compiled page for any entity named in the query, before touching chunks at all - weighted high for summarize-shaped intents, low for causal ones. A page is a synthesis, and when you want a specific arrow you want the edge, not the essay.
• Not zero-embedding. Pages are text. Text embeds. They would go into their own FAISS namespace alongside documents, memories, episodes and events - so the wiki would not replace ANN search, it would give it a better corpus. Searching compiled pages instead of raw chunks is the Tencent result, and it would cost me one namespace rather than an architecture.
• CONTRADICTS as the Error Book. Cortex already extracts CONTRADICTS edges between chunks. That is an Error Book waiting to be used: a contradiction touching an entity is a signal that its page needs recompiling. The graph would detect the error; the wiki layer would repair it.

The why trace would extend without changing shape. Sketched, not shipped:

"why": {
  "strategy": "wiki",
  "page": "Entity:auth-service",
  "compiled_from": <n_sources>,
  "compiled_at": <iso8601>,
  "stale": <bool>,
  "weight": 0.85
}

That last field is the one I care about. A compiled artifact has an age, and an answer read from a page compiled before the incident is a different kind of wrong from an answer that is merely irrelevant. If knowledge compounds, staleness compounds with it - so the trace has to say when the page was written and what it was written from. A wiki without provenance is just a confident summary.

What it costs

None of this is free, and a post that pretends otherwise is selling something.

• Operational surface. FAISS, Postgres, Neo4j, Redis - four stores, four failure modes, four things to back up. What makes it tractable is one responsibility each: FAISS is the only ANN path, Postgres is the source of truth and is never queried for vector search, and per-user isolation is structural rather than a filter bolted on. pgvector came out in migration 0008 precisely because two ANN paths meant neither was authoritative. Adding a wiki layer must not undo that discipline.
• Write-time cost is still cost. Compilation costs LLM calls at ingestion, not query time - you pay once per source instead of once per question, which is the whole bet. But it is a real bill, and it lands before you know whether anyone will ask.
• Over-engineered for a FAQ bot. If your corpus is flat prose and your users ask what-is-this-about, vector-only RAG is correct and all of this is overhead. The argument was never that similarity search is bad.

The failure of vector-only RAG is not a tuning problem. It is a modeling problem - meaning, structure, and time are three different things, and one embedding space can only represent one of them.

But the deeper mistake is architectural, and I made it too. Vector search is a strategy. So is graph traversal, so is a temporal decay, and so - this is the part I got wrong - is a wiki page. None of them is an architecture. The architecture is the layer that decides which of them to trust for this query, fuses them without pretending their scores are comparable, and can tell you afterwards exactly why.

Cortex today: seven strategies, five memory layers, four purpose-built stores, fused by intent, every hit traced. No latency or scale numbers - none are published until measured, and I would rather ship the architecture honestly than the benchmark early. The eighth strategy is next.

But if you take one thing from this, do not take the graph. Take the compile step. You can have it before dinner with a folder, an agent, and git - no cluster, no namespace, no migration - and it will teach you more about the shape of your own knowledge than any k you will ever tune.

The interactive showcase - all seven strategies, the intent-weighted fusion, and the full trace DAG - is at cortex.dezter.com/showcase.

References

• Andrej Karpathy, “LLM Wiki” gist (April 2026) · gist.github.com/karpathy
• Ming, Li, Wu & Que (WeChat, Tencent), “Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki” · arXiv:2605.25480