Corporate context does not belong in the prompt.

A few weeks ago, a colleague from an ongoing insurance project told me what bothered him about the current agent demo. Not the latency. Not the UI. But the moment when the agent provided a fluent explanation for a rejected benefit application - and no one internally could say which rule, which policy and which validity period were binding for it.

The agent had found relevant document sections. He even had the right tone. What was missing wasn't intelligence. It was a place outside the prompt, where the corporate context is binding.

That is exactly the thesis of this article: Corporate context does not belong in the prompt. It belongs in an explicit, versioned, testable model - and only then can an LLM be formulated or an agent suggest an action.

What ends up in the prompt is not context. It's intentional.

In many enterprise setups, the system prompt looks something like this:

Premium customers may be allowed to...
For product group A the following applies…
In the CH region...
Claims over 100,000 require manual review…

That sounds like governance. It's initially just text that a model might pay attention to.

Once the question becomes operational - Why was this application rejected and can it be resubmitted? - the system needs different types of knowledge at the same time:

• The claim belongs to a policy.
• The policy covers certain services.
• The customer has a certain status.
• A service is only refundable under certain conditions.
• Internal guidelines prohibit certain combinations.
• Exceptions only apply to certain regions.
• The answer must be based on the rules in force at the time of the decision.

A classic RAG system can find suitable passages. It does not guarantee that all relevant relationships have been found, inconsistencies will be identified, validity periods will apply, or that the proposed action is permissible. Incomplete retrieval creates incomplete context. This gives rise to hallucinations, non-deterministic behavior and agential decisions that no one can defend.

This is the same diagnosis as Evidence-first AI: The LLM is allowed to formulate. It cannot alone decide what is true, visible and permissible.

The mental model: six questions instead of one search

Before technology names come up, a simple distinction helps:

Control question: An agent uses vector search to find a policy section that manually checks claims over 100,000. What is still missing before the system can reliably block an automatic approval?

No more retrieval. Not MCP. But: the rule as a formal constraint - and the check before the action.

Ontology and Knowledge Graph are not the same thing

These two terms are often mixed up – even in agent demos.

The Ontology is the model of the possible world: Claim is a class. MedicalClaim is a subclass. A claim can reference a policy. Each claim must reference exactly one policy.

The Knowledge Graph contains the concrete facts: Claim 4711 is a MedicalClaim, refers to Policy 815, which has the status Active.

A useful analogy: ontology ≈ grammar and conceptual model; Knowledge Graph ≈ concrete statements in this grammar. Or from a software perspective: ontology ≈ type system plus rules; Knowledge Graph ≈ Runtime Instances.

A property graph with nodes and edges alone is therefore not a strong semantic solution. Without formal meaning, constraints and traceable ownership, it remains just another data structure - and the LLM fills the gaps again from the prompt. This is exactly where the article Knowledge Graph in the company comes in: model and take responsibility for relationships, not guess from the prompt.

Four technical layers – and the context layer

AWS Prescriptive Guidance describes four layers for the Context Ontology Accelerator:

The separation is the real point. A classic agent often mixes everything:

Question → LLM → Vector Search → LLM → Answer

Responsibility is distributed here: determining intent, choosing access to knowledge, creating and controlling queries, retrieving facts, applying rules, explaining results, allowing or denying actions. The LLM remains important. It is no longer the sole decision-making authority.

This maps directly to the context layer:

We also have a glossary and semantic layer: meaning and calculation before relationships and agent access take effect. The accelerator is therefore not a parallel world. It is an AWS interpretation of the same architectural idea - with formal standards where the prompt otherwise only claims “compliance”.

Scan → Model → Serve: the workflow behind the idea

The accelerator calls the process Scan → Model → Serve.

Scan connects sources and makes them readable: schemas, metadata, documents, catalogs. The goal is not yet a complete graph, but rather clarity: Which entities are there? Which relationships are visible? What quality and release problems exist?

Model turns technical metadata into a business model. Ontology Induction means here: The LLM suggests – cust_master could be Customer, tier_cd could be a CustomerTier. Experts check whether this is technically correct. Is it a customer or customer account? Is tier contract or marketing? Does country apply to residence or contract country?

Serve provides context to applications and agents through multiple access types:

• structured questions → SPARQL / Virtual Knowledge Graph
• Relationship and path questions → Graph traversal
• fuzzy semantic questions → Vector Search / GraphRAG
• Admissibility questions → Reasoning and SHACL/OWL

This is the crucial difference to “Vector Search for every question”.

Materialized, virtual – or hybrid

A materialized knowledge graph stores facts physically (e.g. as RDF in Neptune): quickly traversable, good for cross-domain reasoning, but with copy, sync and freshness costs.

A Virtual Knowledge Graph leaves data in the source and maps it (e.g. via R2RML) to the ontology. SPARQL is translated into SQL. Less migration, more dependence on mapping quality and source performance.

In practice, a hybrid architecture is created: operationally virtualizing, materializing what reasoning and integration really need. This is Ontology-Based Data Access – no obligation to “copy everything into the graph” before the first agent works meaningfully. The same logic is behind a hybrid business analytics platform: connect sources and enable decision-making, not migrate everything beforehand.

SHACL as a guardrail – stronger than “please note compliance”

SHACL is particularly interesting for agents: not just data quality, but conditions for action.

Pattern:

LLM suggests action → action is modeled as a testable state → SHACL validated → execute validly / block invalidly and declare violations.

This is the same contract as the Evidence Pack: first evidence and admissibility, then formulation or action. A prompt Please note all compliance rules does not replace this.

MCP is not the knowledge layer in this picture. It is the standardized access protocol: classify intent, create and execute queries, GraphRAG, reasoning. The agent does not need to load the ontology into the context window. It specifically calls up capabilities - under the rights that the control plane enforces (allowlists, denylists, metrics releases). This is the Enterprise difference to many RAG demos: Can the person call the function? and Can this query read this data? are two different questions.

What ontologies don't cure

The statement “ontologies solve hallucinations” is too strong.

You can reduce error classes: incorrect term assignment, inconsistent relationships, violated rules, missing mandatory attributes, logically contradictory results, illegal actions.

They do not automatically solve: incorrect extraction, poor ontology designs, outdated rules, broken mappings, entity resolution errors, incorrect LLM query generation, incomplete sources, or anything outside the modeled domain.

The more realistic formulation:

Ontologies move parts of agent logic from probabilistic prompts into explicit, testable and verifiable models.

That is significant. It is not a universal hallucination cure.

What that means in practice – and what it doesn’t

A realistic adoption model doesn't start with the full accelerator. It starts with a domain question that would be expensive to automate incorrectly - such as the claim question from my colleague's project. For this: clarify terms, model a few relationships, a constraint check before the action, human-in-the-loop in ontology induction, and only then agent tools. This is exactly how we prioritize AI Use Cases for Decision Acceleration: link them to the decision type, not start as a showcase.