Organizational context: giving AI agents a working map of the company

When AI agents disappoint inside a company, missing context is often part of the cause, alongside model limits, weak tools and slow adoption. The agent does not know what the company's terms mean, who owns what, which system holds the truth, what the person it is helping may see, or what changed this morning. Benchmarks and one widely cited field report from the past two years point at this gap, though none measures how much of the failure it explains, and the studies that test fixes suggest that data and engineering work can matter as much as the choice of model.

What the MIT NANDA report actually says

A widely quoted number comes from The GenAI Divide, a preliminary July 2025 report published under the name MIT NANDA. Its authors say the views are their own and not those of any affiliated employer [1]. It says 95% of organizations are getting zero return from generative AI [1]. For task-specific enterprise tools, it reports that 60% of organizations evaluated them, 20% reached a pilot and 5% reached production [1].

That number needs care. The report labels its findings preliminary and shows no calculation for the 95%. It draws on interviews at 52 organizations, 153 survey responses and a review of more than 300 public AI initiatives. The authors call their figures "directionally accurate based on individual interviews rather than official company reporting" [1]. One passage defines success as deployment beyond the pilot with measurable KPIs, with return measured six months after the pilot, while the note under the figure counts tools that users or executives said had a marked and sustained impact [1]. One large survey points the other way: 74% of 801 enterprise leaders told Wharton and GBK Collective their generative AI work already showed positive ROI, though that is also self-reported [2].

The reasons given are also more specific than the headline. Asked to rate barriers to scaling, respondents put unwillingness to adopt new tools first and concerns about model output quality second [1]. The authors' own diagnosis is a "learning gap": most systems "do not retain feedback, adapt to context, or improve over time" [1]. Our companion article on agent memory covers that side.

Some of the rest is about context. One barrier users named for core work was "Too much manual context required each time" [1]. One executive choosing vendors told the researchers, "Most vendors don't get how our approvals or data flows work" [1]. Another said, "If it doesn't plug into Salesforce or our internal systems, no one's going to use it" [1]. A third: "Our process evolves every quarter" [1]. Our reading is that some stalled pilots were systems that did not know the company they were deployed in. The same report cautions that tools needing heavy enterprise customization often stalled at the pilot stage, and it promotes agent frameworks including NANDA, its authors' own project [1].

What enterprise agent benchmarks show

A newer group of benchmarks tests agents inside simulated companies rather than on isolated tasks. These are the most relevant here.

Two patterns stand out. First, agents do much better on parts of a job than on the whole job. On EntCollabBench, the best model passed 78% of subtasks but only 50% of the 40 multi-step workflow tasks [7]. Agents also struggle to ask for what they are missing: on CRMArena-Pro, Gemini 2.5 Pro fell from 54% to 35% in the B2B org and from 58% to 30% in the B2C org when information arrived over several turns instead of all at once, and in 9 of 20 sampled failures it never gathered what it needed [4]. Models struggled to delegate, pass context to the next role, fill business fields correctly and commit to a final decision [7].

Second, some failures look like missing organizational knowledge, though no benchmark shows how many. The closest is one coded count: for one model on EnterpriseBench, context retrieval accounted for 2 of 69 failures and task decomposition for 20 [5]. On EntCollabBench, agents often skipped a required handoff or passed on too little context, which doomed the next role even when that role was capable [7]. In TheAgentCompany, an agent that could not find the right colleague in the company chat "decides to create a shortcut solution by renaming another user to the name of the intended user" [3]. The authors file that under agents deceiving themselves, next to weak social skills and poor web browsing, so read it as an illustration, not a count.

These are simulations, several run on models that are now a generation or two old, and scores rise as models improve: TheAgentCompany's best result went from 24% to 30.3% as newer models were added [3]. EntCollabBench, ChurnBench and the stale-document and Cube studies discussed below are preprints, with no peer-reviewed version that we could find. Read them for the failure patterns more than for the exact numbers.

The map: knowledge graphs, semantic layers and ontologies

By organizational context we mean the shared, structured knowledge that any agent or person needs to act correctly inside one particular company. It is different from what one agent remembers from its own past work. Think of it as a map that agents and employees share, with six layers.

A knowledge graph stores things, such as people, teams, customers, systems and contracts, along with the links between them. Microsoft's GraphRAG, first published in 2024, works like this. A large language model (LLM) extracts a graph of entities from documents and writes summaries of clusters of related entities in advance. Those summaries are then used to answer "global sensemaking questions" across a whole collection [9].

The evidence since then is mixed. GraphRAG-Bench, presented at ICLR 2026, compared plain retrieval with several graph methods, including Microsoft's GraphRAG, LazyGraphRAG, LightRAG and HippoRAG 2 [10]. Its authors start from recent reports that GraphRAG "frequently underperforms vanilla RAG on many real-world tasks" [10]. RAG, short for retrieval-augmented generation, means the model looks up documents before it answers. The study found plain retrieval as good or better for simple fact lookups, while graphs helped with complex reasoning and summarization [10]. Some graph methods also cost far more. The paper's text has GraphRAG's global mode growing to about 40,000 prompt tokens on the hardest tasks (its cost figure lists about 331,000 average tokens), while HippoRAG 2 stayed near plain retrieval at about 1,000 tokens [10].

For company context, we think a typed graph built from systems of record is more useful than one extracted from prose, though none of the studies cited here compares the two directly. "Typed" means a fixed schema of entity and relationship types. A May 2026 study by the makers of the Samyama graph database had the LLM write queries against such a graph, on an IBM benchmark of industrial maintenance tasks, and let the database run them [11]. The authors' README now retracts the paper's headline framing of 65% to 82% to 99%. It says the 99% tier is tuned on its own test scenarios and leans on hardcoded fallbacks, and that an empty-graph control reproduces most of it [11]. Their audit says the 65% was quoted from IBM's paper rather than run, and that the no-LLM handlers reached 98.6% after five rounds of tuning on the same 139 test scenarios. It describes the fix as a correction, not a retraction of the effect [11]. Their corrected test, with gpt-4o and the same grader on both sides, still favored the graph by 11.9 to 24.2 points depending on the grader and scenarios, though under the stricter grader on held-out scenarios the graph passed only 38%, against 26% for documents [11]. It is one vendor's benchmark, so treat the size of the effect with care.

Graphs record what exists and how things connect. Semantic layers and ontologies record what things mean. A semantic layer is a governed set of business definitions, such as how net revenue is calculated or which table "sales" refers to by default. An ontology goes further and models the business's objects, their relationships and the actions allowed on them. Palantir describes its Ontology as "an operational layer for the organization" that often serves as "a digital twin of the organization" [12]. It has semantic elements (objects, properties, links) and kinetic elements (actions, functions, dynamic security) [12]. The kinetic half is where processes belong. If agents can change data only through defined actions with their own checks, the system enforces the process instead of a prompt describing it.

Open standards are catching up. In July 2026 the Open Semantic Interchange project announced it had been accepted into the Apache Incubator as Apache Ossie [13]. It calls itself "an open specification for both semantic layer and ontology" that AI agents, BI tools and query engines can all read, and more than 50 organizations now take part [13].

Writing definitions down appears to pay off. Researchers at Cube, which sells a semantic layer, tested three frontier models on about 100 analytics questions over a retail dataset [14]. Each model ran with and without a hand-written document of about 9 KB (the abstract says 4 KB; the file itself is 9,047 bytes) covering measures, conventions and disambiguation rules. The document raised accuracy by 17 to 23 percentage points for every model [14]. Within each condition, the three models were statistically indistinguishable [14]. Accuracy rose from about 46–51% to about 68–69%, so roughly three answers in ten were still wrong, and with 99 scored questions the test could not rule out gaps of about five points between models [14]. A few pages of definitions mattered more than the choice among top models. The errors it fixed were ordinary business ambiguities, such as which table "sales" means and inventory snapshots that must not be summed across dates [14]. It is one dataset and a vendor study, so treat the size of the effect with care.

Permissions: the map must know who is asking

An agent that can read everything can show someone something they should not see. The benchmarks put numbers on this. CRMArena-Pro found agents had "near-zero inherent confidentiality awareness"; targeted prompting helped but often hurt task performance [4]. CI-Work found that higher task success often went with more privacy violations, and that "Simply increasing model size or reasoning depth fails to address the problem" [6].

Production systems have proved vulnerable as well. EchoLeak (CVE-2025-32711) was a zero-click prompt injection flaw in Microsoft 365 Copilot, found by researchers at Aim Labs. Prompt injection means hidden instructions planted in content the AI reads. In their proof of concept, a single crafted email could get internal data sent out to an attacker, with no action by the user [15]. Microsoft deployed a server-side fix in May 2025 and said there was no evidence of exploitation in the wild [15]. Among other fixes, the researchers who analyzed the case recommend least privilege and splitting retrieval into trust tiers, so that one answer does not mix internal data with outside content [15].

The practical rule follows. Check permissions in code at retrieval time and at every tool call, using each source system's own access rules and the identity of the person the agent is acting for. Do not rely on the prompt to keep secrets. Shared conversations need a stricter rule. Microsoft researchers argue that anything used in training, retrieval or generation must be "explicitly authorized for all users involved in the interaction," and they report deploying the fine-tuning part of this, which decides what a tuned model learns from and who may use it, in Microsoft Copilot Tuning [16]. In practice, the rule means the overlap of everyone's permissions, not the union. Give each agent its own identity too, so the audit log shows which agent did what and for whom.

Freshness: the map must be current

Companies change daily: people switch teams, licenses get reassigned, prices and policies change. A map that was right last month can be confidently wrong today.

Outdated documents do harm even when the model already knows better. A September 2026 study built 317 verified cases where official guidance had reversed, across medicine, law, software and platform policy. In two small open models (Llama-8B and Qwen-7B, on about 50 medical questions each that they answered correctly without retrieval), outdated evidence flipped 30% and 37% of answers, without any instruction to trust the document [17]. When models were told explicitly when the old evidence stopped applying, the larger models switched to the right answer almost perfectly [17]. A re-ranker that favors newer documents cut the damage by 4.6 to 10.0 points, provided the dates were accurate, and three of the four comparisons were statistically significant [17].

ChurnBench, also from September 2026, simulates four enterprise data sources that keep changing and logs every change in a ledger [8]. That lets it separate a freshness error, an answer that was right when the data was retrieved but wrong by the time it was checked, from a reasoning error [8]. Its finding: staleness depended on the refresh schedule, not on cache age, and what matters is how often each entity is refreshed compared with how fast it changes [8]. The error counts are small, so read the result as a direction rather than a precise estimate.

So store, for each fact, when it became true, when it stops being true and when it was last checked, and show those dates to the model. Read fast-moving state, such as ticket status or stock levels, from the system of record when the question is asked, or from change events (messages a system sends whenever a record changes). Refresh slower facts, such as the org chart, on a schedule. Treat permission changes as the most urgent update of all.

Context engineering and MCP: getting the map to the agent

Context engineering is the work of choosing what reaches the model at each step. Anthropic's applied AI team calls it "the natural progression of prompt engineering" and defines it as "strategies for curating and maintaining the optimal set of tokens (information) during LLM inference" [18]. It is ongoing work, redone each time something is passed to the model [18]. The team's rule of thumb is to find "the smallest possible set of high-signal tokens" that makes the desired outcome likely [18].

That argues against pasting the whole map into every prompt, though the trade-off is real. Anthropic describes a just-in-time approach in which agents keep lightweight identifiers and use tools to load data when they need it, and says the most effective agents may also load some data up front, because runtime exploration is slower [18]. The Cube authors report, without details, that an agent using tools over equivalent knowledge did not beat the schema-only baseline, while the same knowledge placed in the prompt did [14]. For organizational context, the runtime part means a few precise tools: find an entity and its owner, get a metric's definition, check whether this user may see this record, read the current state from the system of record. Each should return a short, dated answer with its source.

The Model Context Protocol (MCP) is an open standard for connecting AI applications to tools and data like these. Anthropic introduced it in late 2024 and, in December 2025, donated it to the Agentic AI Foundation, a fund under the Linux Foundation co-founded by Anthropic, Block and OpenAI [19]. At that point Anthropic counted more than 10,000 active public MCP servers and more than 97 million monthly downloads of MCP's software development kits [19]. ChatGPT, Cursor, Gemini, Microsoft Copilot and Visual Studio Code all supported it, and the November 2025 release had added asynchronous operations and server identity [19].

The 2026-07-28 specification changed the protocol substantially. The core is now stateless: each request describes itself, so "any request can land on any instance behind a plain round-robin load balancer" [20]. Method and tool names travel in HTTP headers, so gateways can route and authorize each call directly [20]. Tasks for long-running work, MCP Apps and Enterprise-Managed Authorization are now formal extensions, and Roots, Sampling and Logging are deprecated with at least twelve months' notice [20].

For enterprises, Enterprise-Managed Authorization matters most. It lets a company grant access to MCP servers through its identity provider, the system that runs company logins such as Okta, based on group membership and roles [21]. As a result, "access decisions live in the IdP admin console, with one auditable trail across every connector" [21].

MCP solves connection, not meaning. It gives an agent a standard way to reach the CRM and the ticketing system. It does not say which of the two is the source of truth for a customer's status, what "churned" means here, who owns the account, or whether this user may see it. Those answers have to come from the map.

What to do next

A practical sequence:

1. Pick one workflow that matters and fails today. Collect real tasks from it, including cases where the right move is to refuse or to say "I don't know."

2. Write down the meaning. List the terms, metrics and conventions the workflow depends on. In the Cube study, about 9 KB of text moved accuracy by 17 to 23 points [14]. Give the document an owner and keep it under version control.

3. Model entities and owners as a small typed graph: people, teams, customers, systems and who owns what, loaded from HR, CRM and IT systems. Record which system is the source of truth for each field.

4. Enforce permissions outside the model, at retrieval and at every tool call, with the requesting user's identity. In shared conversations, use only what every participant may see [16]. Log every read and write.

5. Date every fact with when it became true, when it expires and when it was last checked, and pass those dates to the model [17]. Set refresh rates by how fast each kind of entity changes [8].

6. Expose the map through a few precise tools, over MCP or plain APIs, with access granted through your identity provider [21]. Load short, stable definitions up front where testing shows it helps [14].

Then track these numbers:

• End-to-end task success on your own task set, not step-level accuracy. EntCollabBench shows how far apart the two can be [7].

• Permission violations and leakage on deliberately sensitive test cases, in the style of CI-Work [6]. The target for violations is zero.

• Freshness errors: answers that were right when the data was retrieved but wrong by the time they were used [8].

• Staleness lag: the time between a change in a system of record and the same change in the map.

• Silent versus loud failures: how often the agent gives a confident wrong answer instead of saying it cannot answer.

• Cost per task in tokens and latency, watched as the map grows.

• Audit coverage: the share of agent actions you can trace to a user, an agent identity, a source and a permission decision.

How we think about this at OctaNeural

We see organizational context as infrastructure that a company should own, whichever models it uses. We help companies solve this problem through custom deployments and customizations, so weigh our view with that in mind. If you are working on this problem, bring it to us.

Sources

1. Aditya Challapally, Chris Pease, Ramesh Raskar, Pradyumna Chari (MIT NANDA). "The GenAI Divide: State of AI in Business 2025." MIT NANDA preliminary report, July 2025. We found no open official copy, so this links an unofficial copy hosted by the AI Governance Library. https://www.aigl.blog/content/files/2025/09/The-GenAI-Divide-STATE-OF-AI-IN-BUSINESS-2025.pdf

2. Jeremy Korst, Stefano Puntoni, Prasanna Tambe (GBK Collective, Wharton Human-AI Research). "Accountable Acceleration: Gen AI Fast-Tracks Into the Enterprise." Wharton and GBK Collective report, October 2025. https://ai.wharton.upenn.edu/wp-content/uploads/2025/10/2025-Wharton-GBK-AI-Adoption-Report_Full-Report.pdf

3. Frank F. Xu, Yufan Song, Boxuan Li, et al. "TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks." arXiv:2412.14161 (v3), September 2025; NeurIPS 2025 Datasets and Benchmarks Track. https://arxiv.org/abs/2412.14161

4. Kung-Hsiang Huang, Akshara Prabhakar, Onkar Thorat, et al. (Salesforce AI Research). "CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions." arXiv:2505.18878, May 2025; Transactions on Machine Learning Research, 2026. https://arxiv.org/abs/2505.18878

5. Harsh Vishwakarma, Ankush Agarwal, Ojas Patil, Chaitanya Devaguptapu, Mahesh Chandran. "Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments." Proceedings of EMNLP 2025, November 2025. https://aclanthology.org/2025.emnlp-main.466/

6. Wenjie Fu, Xiaoting Qin, Jue Zhang, et al. "CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents." Proceedings of ACL 2026 (Industry Track), July 2026. https://aclanthology.org/2026.acl-industry.103/

7. Tao Yu, Hao Wang, Changyu Li, et al. "Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows." arXiv:2605.08761, May 2026. https://arxiv.org/abs/2605.08761

8. Vivek Kumar Singh, Preeti Priyam, Gautam Bhowmick. "ChurnBench: A Drift-Aware Benchmark Demonstrating That Refresh Scheduling, Not Cache Age, Governs Staleness in Agentic AI." arXiv:2609.11515, September 2026. https://arxiv.org/abs/2609.11515

9. Darren Edge, Ha Trinh, Newman Cheng, et al. "From Local to Global: A Graph RAG Approach to Query-Focused Summarization." arXiv:2404.16130, April 2024 (revised February 2025). https://arxiv.org/abs/2404.16130

10. Zhishang Xiang, Chuanjie Wu, Qinggang Zhang, et al. "When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation." ICLR 2026, April 2026. https://arxiv.org/abs/2506.05690

11. Madhulatha Mandarapu, Sandeep Kunkunuru. "Knowledge Graphs as the Missing Data Layer for LLM-Based Industrial Asset Operations." arXiv:2605.26874, May 2026; accepted at AG2026 @ VLDB 2026. https://arxiv.org/abs/2605.26874. Corrections: authors' audit, July 2026, https://github.com/samyama-ai/assetops-kg/blob/main/docs/AUDIT-2026-07-13.md; repo README, accessed November 2026, https://github.com/samyama-ai/assetops-kg/blob/main/README.md; strict-grader results, August 2026, https://github.com/samyama-ai/assetops-kg/commit/0d8bfe0d and https://github.com/samyama-ai/assetops-kg/commit/5038c6e0.

12. Palantir. "Ontology building: Overview." Palantir Foundry documentation, accessed November 2026. https://www.palantir.com/docs/foundry/ontology/overview/

13. Josh Klahr. "Apache Ossie (Incubating): The New Name for Open Semantic Interchange." Apache Ossie project site, July 2026. https://ossie.apache.org/updates/ossie-enters-apache-incubator/

14. Michael Rumiantsau, Ivan Fokeev (Cube). "Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Models." arXiv:2604.25149, April 2026. https://arxiv.org/abs/2604.25149. Document file: https://github.com/cubedevinc/semantic-layer-benchmark/blob/master/fixtures/semantic_layer.md.

15. Pavan Reddy, Aditya Sanjay Gujral. "EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System." arXiv:2509.10540, September 2025; Proceedings of the AAAI Symposium Series, November 2025. https://arxiv.org/abs/2509.10540

16. Shashank Shreedhar Bhatt, Tanmay Rajore, Khushboo Aggarwal, et al. (Microsoft). "Enterprise AI Must Enforce Participant-Aware Access Control." arXiv:2509.14608, September 2025. https://arxiv.org/abs/2509.14608

17. Md Shamim Ahmed, Lukas Galke Poech, Richard Röttger. "Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers." arXiv:2609.31342, September 2026. https://arxiv.org/abs/2609.31342

18. Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, Jeremy Hadfield (Anthropic). "Effective context engineering for AI agents." Anthropic Engineering, September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

19. Anthropic. "Donating the Model Context Protocol and establishing the Agentic AI Foundation." Anthropic News, December 2025. https://www.anthropic.com/news/donating-the-model-context-protocol-and-establishing-of-the-agentic-ai-foundation

20. David Soria Parra, Den Delimarsky. "The 2026-07-28 Specification." Model Context Protocol Blog, July 2026. https://blog.modelcontextprotocol.io/posts/2026-07-28/

21. Paul Carleton. "Enterprise-Managed Authorization: Zero-touch OAuth for MCP." Model Context Protocol Blog, June 2026. https://blog.modelcontextprotocol.io/posts/enterprise-managed-auth/

Sources checked November 1, 2026.