Why RAG will never be a corporate brain

Almost all enterprise AI today is built around one idea. Collect the documents. Build a vector index. Plug in an LLM. Get answers.

This works beautifully for finding information. It helps remarkably little with making decisions. It is worth being precise about why, because the reason is structural, not a matter of tuning.

A concrete case: customs classification

Take a customs authority. Tens of thousands of documents: statutes, ministerial orders, the tariff nomenclature and its explanatory notes, international agreements, binding tariff information, case law, internal instructions.

Suppose the model reads all of it. Every page, perfectly retrieved, in context.

It still does not know:

• What outranks what. A ministerial order does not override a treaty obligation. An internal instruction does not override either. Precedence is a property of the legal system, not of the text.
• What is still in force. Documents do not usually announce their own repeal. The amendment lives in a different document, published later, sometimes in a different register.
• What the scope is. A rule about "textile articles" applies to a specific set of tariff headings, and knowing which set requires the nomenclature, not the sentence.
• Where the contradictions are. Two provisions that conflict look identical to cosine similarity. Resolving them requires knowing the rule that governs conflicts.
• What has been decided before. Consistency with prior rulings on materially similar goods is often the actual legal requirement — and "materially similar" is a domain relation, not a text distance.

An expert handles all of this without effort, and not because they have memorised more documents. They have an internal model of how the domain is organised: which entities exist, how they relate, which relations imply which obligations, what beats what.

That model is what retrieval systems do not have. It is what an ontology is.

Ontology, in the engineering sense

The word carries academic baggage, so let us be concrete. In this context an ontology is a formal, machine-checkable model of a domain:

• the entities that exist — goods, declarations, parties, procedures, permits;
• the relations between them — is-a, part-of, supersedes, requires, excludes, applies-to;
• the constraints — a declaration in this procedure requires that permit; this classification excludes that one;
• the provenance — every assertion traceable to the source that establishes it, with its version and its validity period.

None of this is exotic. Every serious regulated organisation already has fragments of it: classifiers, registries, reference data, decision trees in a compliance manual, the tacit rules a senior colleague teaches a junior one. It is simply not written down in a form a machine can reason over — so it is exactly the knowledge an AI Factory silently drops on the floor.

Note what the ontology gives you that embeddings cannot. Vector similarity is a single, undifferentiated, symmetric notion of "relatedness". A domain runs on many typed, directed relations, and the difference between "these two rules are similar" and "this rule supersedes that one" is the difference between a search result and a decision.

What changes when the structure exists

With a domain model in place, a generated answer becomes something you can interrogate.

An LLM proposes a classification. The symbolic layer can now check whether the proposed heading actually exists in the current nomenclature version, whether its exclusion notes conflict with the goods description, whether the required permit is present, whether a prior binding ruling covers a materially similar item — and either confirm the proposal, reject it with a reason, or route it to a human because the case sits in a region where the rules are genuinely ambiguous.

Three properties appear that pure retrieval cannot produce:

Explainability. Not "the model attended to these tokens", but a chain: this heading, because of this note, which is in force since this date, consistent with this precedent. An explanation a domain expert can disagree with — which is the only kind worth having.

Reproducibility. The same case yields the same decision, because the deciding part is not sampled from a distribution. The model's creativity is confined to proposing; the accepting is deterministic.

Boundedness. The system can be prevented from asserting things the domain model says are impossible. Not by asking the model nicely in a prompt, but because the assertion fails a check before it becomes an answer.

This is not "RAG is useless"

It is not. Retrieval is the right tool for grounding language in your corpus, and it stays in the architecture. The claim is narrower and firmer: retrieval is a component, not an architecture. A system whose top-level design is "retrieve, then generate" has no place to put precedence, validity, authorisation, or proof — so those things end up in prompt strings, where they are unenforceable.

The alternative is not to abandon language models. It is to give them a counterpart. Neural for language, extraction, hypothesis and interaction; symbolic for structure, constraint, precedence and proof. Systems built on that pairing are called neuro-symbolic, and we think that is where credible enterprise AI is going.

Which raises the practical question: what actually runs that loop? Something has to hold the model, the ontology, the policies, the tools and the human in one execution flow.

---

Next in this series: What a harness system is — the architecture that sits between an agent and reality.