Enterprise AI context engineering: The layer that governs what agents see

1. The failure nobody logs

Most enterprise AI agents store no knowledge internally. They fetch documents from enterprise systems at the moment a question arrives, a step the field calls retrieval. The failure below shows what happens when that step runs without governance.

A claims agent denies a customer’s reimbursement. The determination is wrong. The insurer revised the policy governing that claim three months earlier, and the revision reverses the exclusion the agent cited. The trace shows why. Both policy versions lived in the vector store as embedded chunks. At query time, the retrieval pipeline scored them on cosine similarity against the claim text. Cosine similarity measures how closely two passages match in wording. The superseded version scored marginally higher because its wording matched the claim’s language more closely. The index carried no metadata for document authority, effective date, or supersession, so the ranker had nothing else to use. Ranked purely on similarity, the wrong policy won, and the model reasoned flawlessly over it. A ranker that reads only wording cannot know that one document canceled the other.

Nothing in this failure implicates the model. The error occurred upstream, in the step where the enterprise assembled what the agent would see. In most organizations, that step has no owner, no architecture, no quality gates, and no audit trail. This article names that step and describes the layer that should govern it.

2. From prompt craft to context discipline

The engineering community has a term for this problem. Anthropic defines context engineering as finding the configuration of context most likely to produce the model’s desired behavior. It treats context context as a critical but finite resource [1]. Context here means the full token set present at sampling time. A token is a small text fragment, roughly a word, that the model processes as a unit. The set includes system instructions, tool definitions, retrieved passages, tool results, and message history. Prompt engineering optimizes the wording of one component, while context engineering optimizes the composition of all of them.

Practitioners have treated this as a developer concern. For enterprises, it is a governance concern. When agents approve payments, route claims, and answer regulators, context assembly becomes a control point. Data pipelines went through a similar shift a decade ago. An enterprise that cannot reconstruct what its agents saw when they acted cannot defend those actions afterward.

3. Why bigger windows made it worse

The intuitive fix was capacity. Agents fail because they lack information, the reasoning went, so give them room for more of it. Model vendors obliged. Context windows, the total text a model reads at once, grew from thousands of tokens to millions. The failures persisted, but in a different form.

Liu and colleagues showed that language models use long contexts unevenly. Accuracy peaks when relevant information sits at the beginning or end of the window and degrades significantly in the middle [2]. Position determines whether the model uses information at all. Chroma's evaluation of 18 models documented a second effect its authors call context rot. Performance degrades as input length grows, even on tasks the models handle reliably at shorter lengths [3]. Degradation accelerates when the window contains distractors semantically close to the answer. Past a threshold, additional context becomes noise that suppresses the signal it was meant to strengthen.

For enterprises, the implication is uncomfortable. Teams that respond to agent errors by loading more documents into the window amplify the failure mode. The binding constraint is the model's attention across the window, and attention thins as the window fills. Managing that constraint requires selection, and selection needs an owner with criteria. That is an architectural function.

4. Retrieval solves a narrower problem

Most enterprises believe they solved context when they deployed retrieval-augmented generation (RAG). RAG grounds model outputs in enterprise documents by fetching semantically similar passages from the company’s own stores at query time [4]. It is necessary infrastructure, but an incomplete control.

Retrieval answers one question: which stored passages best match this query? Context engineering answers a broader one: what should this agent see, from all available sources, given what it is about to do? The gap shows up in production. A similarity ranker has no signal for supersession, so the stale policy from the opening example outranks the current one. It applies no entitlement check at query time, so a low-privilege request can surface passages its principal lacks authorization to open. It imposes no size discipline on tool results, so 10,000 tokens of log output crowd the window after a single call. Retrieval is one component inside the context layer. Treating the component as the layer is how enterprises end up with grounded, well-cited, but wrong answers.

Parts of this layer already exist as commercial tooling. Vector databases support metadata filters that enforce effective-date and authority constraints at query time. Observability platforms log what entered the window and flag anomalies. Prompt firewalls screen retrieved content for injection payloads before it reaches the model. Enterprises can assemble the layer from these components. Ownership is what the tooling cannot supply. The components must run as one layer, under one policy framework, instead of as an ad hoc collection per project.

5. Four capabilities the layer must provide

A governed context layer performs four functions. Figure 1 shows how they sequence between enterprise sources and the model call.

Figure 1. The context assembly pipeline

Source: Infosys

6. Context governance in multistep agent workflows

The four capabilities describe the layer’s work at a single decision point. Production agents rarely stop at one step. They plan, call tools, spawn subagents, and accumulate context across turns. In these pipelines, the governance burden compounds.

Each tool call returns output that re-enters the window. Without continuous budget enforcement, a chain of five calls fills the window before the agent reaches the step that demands reasoning. Each subagent in a multiagent workflow inherits the context its orchestrator passes down, and every handoff is an assembly step. A handoff that carries stale or unauthorized content propagates the error. An unlabeled handoff launders untrusted content into a trusted position. Provenance carries the audit burden here. The final agent’s context may hold content that crossed three intermediate steps. The trail must trace each element to its origin system.

The design implication follows. Budget compression, scoping checks, and provenance tagging apply at every step where new content enters the window, tool results included. Governing only the first retrieval leaves the rest of the chain exposed. Enterprises that let tool results accumulate unmanaged watch their pipelines degrade the way single-shot agents did. The model performed as designed while the context around it grew ungoverned.

7. The same claim, governed

Return to the claims agent that denied the reimbursement. Under a governed layer, the same query takes a different path. Selection filters on effective date and supersession metadata, so only the current policy is eligible to enter the window. Scoping confirms the agent, acting for this service context, may read it. Provenance labels the passage with version, effective date, and trust tier, and the agent’s answer carries the citation. Budget places the policy text where the model attends best and trims claim history around it. The agent approves the reimbursement, and the trace shows exactly why. The same model and the same claim produce the opposite outcome. The only change is what the agent saw.

8. Context is the new injection surface

The security case for the layer is now explicit. The Open Worldwide Application Security Project (OWASP) ranks prompt injection as the top risk for large language model (LLM) applications and highlights the indirect form. Instructions arrive embedded in documents, websites, and emails, and the model processes them as if they were legitimate [5]. Every document an agent reads is executable input. The attack targets the assembly step, which placed untrusted content beside trusted instructions with nothing marking the difference.

This reframes context engineering as perimeter defense. The layer's provenance capability enables segregation. Content from external sources arrives labeled and stays quarantined from instruction-bearing positions. Policy strips instruction-like payloads where required. Enterprises that let any retrieved byte flow unmarked into the window have built agents that will eventually take orders from a PDF.

9. Who owns the layer

The context layer needs assigned ownership, and the assignments map to functions enterprises already have. Data governance owns selection policy because authority ranking between sources is a data stewardship decision. Identity and access management owns scoping because deciding what an agent may see on a user’s behalf is an access-control decision. Security owns provenance and segregation controls because indirect injection is an attack surface. Platform engineering owns budget policy and the assembly pipeline itself.

Service levels for context quality should mirror the discipline of data quality: Each owner tracks the metric its capability defines, reported per agent.

One test settles whether the enterprise governs the layer. When an agent acts on stale, unauthorized, or poisoned context, someone's dashboard should have shown it first. An enterprise without that dashboard is running agents in production and context assembly on improvisation.

10. What good governance looks like and where to start

A governed context layer is measurable, and the four capabilities already define the key metrics. Selection tracks stale-context rate, scoping tracks leakage rate, provenance tracks reconstruction coverage, and budget tracks tokens per completed task. These four numbers, reported per agent, form the minimum instrumentation. An enterprise that cannot produce them has no evidence the layer works.

For teams starting from zero, scoping delivers the highest return first. Closing the authorization gap at query time yields an immediate compliance and security benefit. It needs little new infrastructure and creates no model dependency, because most identity systems already expose the required access-control primitives. Selection discipline comes second, led by effective-date filtering, because it addresses the failure class in the opening example. Provenance tagging and budget policy follow as the pipeline matures.

Small engineering teams should stage the work instead of building all four capabilities at once. A hardcoded effective-date filter in production prevents more failures than a provenance system still on the roadmap. Governance at small scale means explicit policy decisions, even under manual enforcement. Knowing which source wins when two documents conflict is worth more than sophisticated tooling with no policy behind it.

11. The layer decides the ceiling

Model capability sets the ceiling on what agents can do. Context quality sets the ceiling on what they actually do. The second ceiling is lower in almost every enterprise deployment, and it is the one enterprises control. Governing what agents see costs less than retraining models and moves faster than the next release. It is also auditable in ways model internals never will be. The enterprises that treat context assembly as architecture will run more reliable agents on the same models their competitors use. The rest will keep filing model bugs against failures the model never caused.