The Missing Piece: Why Billion-Dollar AI Projects Keep Failing — and the Invisible Infrastructure Race to Fix It

Enterprises have poured staggering sums into artificial intelligence. The models are brilliant. The demos dazzle boardrooms. And then, with remarkable consistency, the projects collapse. Over 80% of enterprise AI initiatives fail before reaching production. Forty-two percent of companies abandoned most of their AI efforts in 2025, discarding nearly half their proof-of-concepts along the way. Only 5% of generative AI pilots have delivered measurable revenue. These are not the teething troubles of an immature technology. They are symptoms of a structural gap that the industry is only now learning to name.

The problem is not intelligence. It is ignorance — specifically, the AI’s ignorance of the organization it is supposed to serve.

Think of it this way. You could hire the most brilliant consultant on the planet, but if you dropped them into your company with no access to your processes, your institutional history, your rules, or even your jargon, they would flounder. That is precisely the situation most enterprise AI agents find themselves in. They have raw reasoning power but lack the organizational context to wield it effectively. The emerging discipline attempting to solve this is called context engineering, and the infrastructure being built around it — Context-as-a-Service, or CaaS — may be the most consequential and least discussed layer of the AI stack.

The backstory begins with Retrieval-Augmented Generation, or RAG, the workhorse approach that became the default method for grounding AI outputs in real data. RAG fetches relevant documents and feeds them to the model alongside each query. It was a pragmatic fix for hallucination, and it worked — up to a point. Stanford researchers found that even specialized legal AI tools still hallucinated in 17% to 34% of queries. RAG pipelines introduced compounding latency. Worst of all, RAG was designed for a static question-and-answer world, not for the autonomous, multi-step decision-making that agentic AI demands.

Two failure modes tell the story. “Context pollution” overloads an AI agent with irrelevant information until it drowns in noise. “Context rot” describes how an agent’s grip on useful information degrades as conversation histories and retrieved documents pile up — a phenomenon researchers call being “lost in the middle,” where data buried in the centre of a long context window effectively becomes invisible to the model. Neither failure reflects a deficit of reasoning. Both reflect a deficit of curation.

This is the insight that crystallized in mid-2025 when Shopify’s CEO and prominent AI researcher Andrej Karpathy independently championed context engineering as a distinct discipline. The realization was blunt: most AI pilots fail not because the models are not smart enough, but because they do not know enough about your business. Research by Cognizant confirmed the stakes — context-aware AI agents achieved three times higher accuracy and 70% fewer hallucinations than baseline deployments.

The technical infrastructure is coalescing fast. Anthropic’s Model Context Protocol, or MCP, introduced in late 2024, has become a de facto standard for connecting AI systems to external data. Within a year it attracted adoption from OpenAI, Google DeepMind, Salesforce, ServiceNow, and dozens of others, racking up over 97 million monthly SDK downloads. Anthropic subsequently donated MCP to a Linux Foundation initiative backed by Microsoft, Google, and AWS. For context providers, MCP solves the brutal integration math: build once, serve any compatible AI application.

Alongside MCP, knowledge graphs are providing the semantic scaffolding that flat document retrieval cannot. Where RAG fetches fragments, knowledge graphs capture relationships between entities, enabling the multi-hop reasoning that real organizational questions demand. Google’s Agent-to-Agent Protocol complements MCP by governing how AI agents from different vendors talk to each other. Together, these standards are assembling the plumbing of an interoperable agentic ecosystem.

The market riding on this plumbing is enormous. The AI agent market, valued at $7.8 billion in 2025, is projected to reach $52.6 billion by 2030. Foundation Capital frames the broader opportunity as a $4.6 trillion shift from Software-as-a-Service to “Service-as-Software,” where AI does not just provide tools but delivers outcomes. CaaS platforms are the institutional memory that makes those outcomes possible.

The competitive landscape is wide open. Startups like Workfabric AI, Contextual AI (which has raised $100 million from backers including Nvidia and HSBC), and memory-layer providers Zep and Mem0 are staking early claims. Enterprise architecture vendor Ardoq has repositioned its organizational blueprints as ready-made AI context. Hyperscalers are building horizontal enablers. Cognizant has committed to deploying 1,000 context engineers — a new job title that barely existed a year ago.

The vertical opportunities are where things get particularly interesting. In law, context means preserving decision traces — the reasoning behind strategic choices that generic systems discard. In healthcare, Abridge’s Contextual Reasoning Engine converts clinician-patient conversations into structured clinical notes with auditable links back to source audio, addressing the trust gap that keeps doctors skeptical. In construction, startups are ingesting everything from weather data to building information models to provide real-time project intelligence. Financial services, where banks report 77% returns on agent deployments, need context that spans shifting regulations across jurisdictions.

The risks are real. Over 40% of agentic AI projects could be cancelled by 2027 as costs and complexity bite. Large enterprises may choose to build context infrastructure internally. Security researchers have flagged MCP vulnerabilities including prompt injection and data exfiltration. And the talent pool for context engineering — a discipline blending domain expertise, data engineering, and AI systems design — is vanishingly thin.

Yet the core thesis is hard to argue with. The AI models are already powerful enough. What they lack is knowledge of the world they are being asked to operate in. The winners of the next phase of enterprise AI will not be determined by who has the smartest model, but by who has the richest, most defensible organizational context. That is not a speculative bet on future technology. It is a fix for a problem that is costing enterprises billions right now.

Read our full Report Disclaimer.