Your AI Agents Are Not Reading Your Semantic Layer: Where The Business Context Actually Comes From

Posted on by
Contents show
An ink and watercolour illustration of several thin lines converging into a single vessel, with one line passing through a fine mesh gate, representing the routes by which AI agents reach the semantic layer and the routes that bypass it

A confidently wrong answer from an AI agent usually has a boring cause. The model did not misread the question. It never knew what your organisation means by revenue, which customer record counts as current, or whether a churned account includes the one that paused for a month.

The industry has a name for the thing that carries those meanings: the semantic layer. Some teams call the surrounding discipline context engineering, and that label is a fair description of the work, which is deciding what a model sees before it answers. The governed semantic layer is the part of that work that produces a number rather than a paragraph. Most data teams either have one or are building one. The problem this article is about is that having a semantic layer does not mean your agents read it, and the ways agents get context instead are convenient, cheap, and silent about what they leave out.

This is an argument from mechanism rather than from a survey, and the distinction matters. There is no independent study I could verify that measures where enterprise agents source business context, so what follows comes from what the vendors of this infrastructure document, from Gartner's published predictions, and from the mechanics of the protocols agents use every day.

1. What a governed semantic layer actually is

A semantic layer sits between your data and everything that consumes it, and its job is to turn physical structure into business meaning. Databricks describes it as the architectural component that translates source data into shared business meaning, and its account of the components is the clearest short version available:

  • Measures, which quantify outcomes as computed functions and stay independent of grouping.
  • Dimensions, the axes of analysis such as customer segment, region or fiscal period.
  • Joins and relationships, declared explicitly so that lineage is visible rather than re-coded in every query.
  • Filters, which encode business rules inside the metric definition, so that active contracts only, or last 90 days, or exclude test accounts become part of the metric's identity rather than an afterthought in a report.
  • Metadata and governance, meaning ownership, descriptions, certification status, synonyms, lineage, row level security and column masking, which travel with each definition.
  • Performance and caching, where materialisation strategies serve common measure and dimension combinations from precomputed results.

The important word in that list is travel. The value of a governed semantic layer is not that definitions exist somewhere. It is that a definition arrives at the point of use with its filters, its access rules and its certification attached, whether the consumer is a dashboard, a notebook or an agent.

The historical reason this is fragile is that semantics used to live inside business intelligence tools. Each tool had its own modelling language, so definitions diverged, engineers maintained the same logic twice, and, in Databricks' phrase, the right number depended on where you asked the question. Modern approaches move semantics into the data platform or into a headless service that exposes them through APIs, which is what makes an agent able to use them at all.

2. The three paths agents take instead

When an agent needs business context, the governed semantic layer is one option among several, and it is rarely the path of least resistance. Three alternatives dominate, and each one drops something specific.

Retrieval over documents. This is the pattern most teams build first, and we covered the mechanics in our guide to retrieval augmented generation. An agent embeds a question, retrieves passages, and answers from them. It works well when the answer lives in prose. It fails at definitions, because the definitive version of a metric is a calculation with filters and joins, not a paragraph. A semantic layer expressed only as documentation becomes, in this path, a document that may or may not be retrieved, and that a model may or may not follow.

Direct queries against live systems. Here the agent writes SQL, calls an API, or invokes a tool on a Model Context Protocol server and reads whatever comes back. This is the pattern with the strongest pull, because it is the simplest to wire up. MCP is an open standard for connecting AI applications to external systems, and its own documentation describes the scope plainly: local files, databases, tools such as search engines and calculators, and workflows such as specialised prompts. Enterprise chatbots connecting to multiple databases across an organisation is listed as a normal use case.

Nothing in that description requires a governed definition. A direct query returns what the table contains, and the column names remain the database's rather than the business's. If the tool surface exposes base tables, the agent will use base tables, because that is what it was given.

Long context loading. Pasting a large set of documents or records into the model's context window moves the problem rather than solving it. The model receives raw material with no agreed definition attached, and the cost profile is different too, which is the part most teams notice first. Our guide to inference caching covers why stable prefixes matter when the same context is resent, and why reshuffling that context destroys the discount. A definition that arrives as part of a stable, certified block of context is cheaper and more reliable than one that arrives as part of a pasted pile.

None of these three paths is wrong. Each of them can sit in front of a semantic layer, and in the strongest implementations they do. The problem is that the default configuration of all three leaves the governed layer out.

3. Why the layer and the agent drift apart

Four forces push agents away from the semantic layer, and understanding them is what makes the fix tractable.

Tool surfaces are easier than semantics. Connecting an agent to a database takes an afternoon. Curating and exposing governed metrics takes a data team, a definition debate and a change process. Teams under delivery pressure wire up the database and intend to add the semantic layer later, which is how a temporary shortcut becomes the architecture. The semantic layer then lives in a different system from the one the agent queries, and nothing in the agent's configuration tells it otherwise. The existence of products whose selling point is making a semantic layer queryable in natural language, such as the dbt MCP server, is itself evidence that being queryable is not the default state.

Semantics historically lived in the wrong place. When definitions live inside a business intelligence tool, agents cannot see them at all. Databricks' account of the shift from tool-bound modelling languages to platform-native semantics is a description of a migration most organisations are still in the middle of, and until it finishes the agent has nothing governed to read.

The naming problem is genuinely hard. dbt Labs describes semantic precision as a new and scarce skill: writing metric definitions and documentation that are unambiguous to a human reader and to a model reasoning over them. An ambiguity that a human resolves with context, such as whether revenue is recognised at the order line or the order, is exactly the ambiguity a model resolves with a guess.

The failure is quiet. A wrong number from a dashboard triggers a reconciliation meeting, because someone notices. A wrong number from an agent arrives inside a fluent paragraph, and the fluency is the problem. Databricks puts it well in describing what happens without a semantic layer: the model generates plausible sounding queries that may be subtly or significantly wrong, and presents the result with equal confidence.

Gartner's published research gives the financial shape of this, and one figure in it is measured rather than forecast. In a survey of 1,203 data management leaders, 63 percent either did not have, or were unsure whether they had, the right data management practices for AI. On top of that, the firm predicts that through 2026, organisations will abandon 60 percent of AI projects that are unsupported by AI-ready data, and separately that more than 40 percent of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls. Business context sits underneath all three of those causes. The same research argues that AI-ready data is not a one-off project, which is the point that matters here: definitions drift as the business changes, so the semantic layer needs an owner rather than a launch date.

4. How to find out which path your agent takes

This is a diagnostic you can run this week, with no new tooling. Six questions, and the answers are observable.

  1. Read the tool call log and name every context source. For a representative set of agent runs, list where each fact came from. If the majority resolves through documents, direct queries or pasted context, you have measured the gap.
  2. Ask the agent to define a disputed term. Pick a term your organisation argues about, such as an active customer. If the answer paraphrases a document, the definition is not in the path. If it cites a governed metric with its filters, it is.
  3. Take one number and reconcile it two ways. Run the same question through the agent and through the governed report. A mismatch is a routing problem, not a model problem.
  4. Check whether raw SQL is exposed. If the agent can write against base tables, some fraction of its answers will, and no prompt will reliably stop that.
  5. Inspect your MCP server's surface. Look at what it publishes. If it publishes tables, the agent sees tables. If it publishes metrics with descriptions, owners and certification, the agent sees business meaning. Nvidia's work on enforcing the boundary around agent access, including inspection of Model Context Protocol traffic, is relevant here, because the tool surface is now a security boundary as well as a semantic one. We covered that runtime in the Nvidia OpenShell article.
  6. Check whether answers carry provenance. An agent that can cite which definition produced a number is an agent that read the semantic layer. An agent that cannot is guessing in a confident voice.

5. How to put the semantic layer in the agent's path

The pattern that works is smaller than a platform project. Make the semantic layer the easiest thing for the agent to reach, and make the raw alternative slightly harder.

Expose metrics as tools, not tables. A tool named with the business term, carrying its definition, its filters and its owner, gives the model something to choose deliberately. dbt's approach of making its semantic layer queryable in natural language is one implementation of this idea, and Databricks describes the same behaviour in two steps: grounding, where the agent reads descriptive context about available metrics and governance rules before answering, and execution, where it queries the governed interface instead of generating raw queries against base tables.

Keep raw access available but privileged. Analysts need SQL. An agent answering a revenue question does not, and the difference is a policy decision rather than a technical limit. A semantic layer that is reachable but optional loses to the table that is reachable and immediate. This is the enforcement question we covered in the guide to building production grade agent systems, and it is the same boundary question that decides how much autonomy an agent should have in the first place.

Add what generative consumers need. Definitions alone are not enough for a model. Databricks names the additional metadata that matters: synonyms and acronyms that map colloquial terms to canonical fields, display rules such as currency formatting, example queries that teach common question shapes, and domain instructions that scope interpretation.

Version definitions and check the blast radius before changing one. Semantics as code, with review and lineage, is what turns a definition change from an incident into a deployment. The same discipline shows up in how platform teams choose tools, which our survey of agent orchestration platforms records from the adoption side.

Give the layer an owner. dbt Labs frames context provision as a distinct responsibility, alongside system design and governance ownership, and notes that analytics engineers are often not trained for it explicitly. Teams that assign the work explicitly get definitions that models can use. Teams that treat it as documentation get paragraphs.

6. What to measure once the layer is in the path

The metrics that matter are the ones that connect agent behaviour to business meaning.

  • The share of agent answers that resolve through the semantic layer's governed definitions, measured from the tool calls rather than from self-report.
  • Term confusion incidents, counted when an agent applies one internal meaning of a term where another was intended. These are the raw material of confidently wrong answers.
  • Definition change impact, meaning how many agent surfaces, dashboards and downstream metrics a change touches. This is what lineage buys you, and it is the difference between a same day change and a two week investigation.
  • Root cause tagging on wrong answers, separating a model reasoning failure from a context supply failure. Most teams cannot currently make that distinction, which is why the same incident recurs.

The direction of travel is not in doubt. If agents are going to answer business questions, the definition of the business has to be available to them at the moment of the answer, with its filters, its access rules and its owner attached. Everything else is a decision about where to spend the engineering, and the honest summary is that the governed semantic layer is the cheap option once the cost of one confidently wrong number is priced in.

Where the numbers stop

  • There is no independent survey I could verify that measures where enterprise AI agents source their business context. The figures in this article come from analyst predictions and from vendors of the infrastructure in question, and the article is an argument from mechanism rather than a measurement of the market.
  • Databricks sells platform-native semantics and dbt Labs sells a metrics layer and a semantic layer product, so both have a commercial interest in the framing that semantics should be governed and central. Their technical descriptions of the components and failure modes are consistent with each other, which is why they are used together here.
  • Gartner's 60 percent and 40 percent figures are predictions with expiry dates, made in 2025 and applying through 2026 and 2027. They describe risk rather than observed outcomes, and the 60 percent figure is specifically about projects unsupported by AI-ready data rather than about semantic layers as such. The 63 percent figure comes from a Gartner survey of 1,203 data management leaders, and it measures data management practice rather than agent behaviour.
  • The 72 percent figure for AI-assisted coding comes from dbt Labs' own 2026 State of Analytics Engineering report, as cited in the company's blog. It describes how analytics engineers work, not where agents get context.
  • I have not run a controlled test comparing agent accuracy with and without a governed semantic layer. No vendor citation in this article should be read as my own measurement.

FAQ

What is a semantic layer in simple terms?

A semantic layer is a governed translation between your data and the business meaning of that data. It defines metrics, dimensions, joins, filters, ownership and access rules in one place, so that every consumer, whether a dashboard, a notebook or an AI agent, gets the same answer and the same definition. It is what makes the business term the reliable handle rather than the table name.

Why do AI agents ignore the semantic layer?

Usually because nothing connects them to it. Agents reach for whatever tools are exposed, and the easiest tools to expose are databases, APIs and documents. A semantic layer becomes visible to an agent when it is published through that tool surface, with descriptions the model can read, and when the raw alternative is limited rather than unlimited. Closing that gap is the practical half of context engineering. Without those two conditions, the agent answers from the material it can reach, which is often technically correct and business blind.

Does retrieval augmented generation remove the need for a semantic layer?

No, and the two solve different problems. Retrieval finds relevant material, which works well for prose and poorly for definitions, because a definition is a calculation with filters rather than a passage. The dependable pattern is to keep the governed definition as the source of the number and use retrieval for everything around it, including the explanation, the background and the caveats.

The short version

Most organisations now have a semantic layer or are building one, and that does not decide what an agent reads. Agents take business context from documents, direct queries over SQL, APIs and MCP servers, or context pasted into the prompt, and all three paths leave the governed definition out unless somebody puts it in. The tell is easy to find: read the tool call log and see whether a fact resolved through a governed metric or through whatever the agent could reach. The fix is smaller than a platform migration. Publish metrics as tools with descriptions, owners and certification rather than exposing base tables, let the platform carry access rules and lineage with each definition, keep raw SQL available but privileged for agents, and tag wrong answers by root cause so a context supply failure stops being mistaken for a reasoning failure. Gartner predicts that organisations will abandon 60 percent of AI projects unsupported by AI-ready data through 2026, and business meaning is the part of readiness that gets built last.

Sources
Subscription Form