Every founder I know has spent the last two years getting better at prompting ChatGPT, Claude, or some new AI agent. That was the right instinct in 2023. It's the wrong one now. In June 2025, Andrej Karpathy - the researcher who helped build OpenAI and later ran Tesla's AI team - coined a new term for what actually separates AI systems that work from ones that don't: context engineering. Not the words you type into a chat box, but the documents, memory, and tool outputs you deliberately feed into the system before it ever has to think. The founders getting real ROI from AI in 2026 aren't writing cleverer prompts. They're building the information pipelines around their AI tools with the same rigor they'd apply to hiring.
- In 2025, Anthropic defined context engineering as "the set of strategies for curating and maintaining the optimal set of tokens during LLM inference," warning that models suffer "context rot" as irrelevant information piles up (Anthropic Engineering Blog, September 2025)
- MIT's NANDA initiative found that 95% of enterprise generative AI pilots fail to produce measurable P&L impact - and traced the cause to an organizational "learning gap," not model quality (MIT NANDA, "The GenAI Divide," August 2025)
- Gartner projects task-specific AI agents will appear in 40% of enterprise applications by the end of 2026, up from under 5% in 2025 - while separately predicting over 40% of agentic AI projects will be canceled by 2027 (Gartner, 2025)
- A Chroma Research study testing 18 frontier models found every one of them degraded in accuracy as input length grew, often well before hitting the model's stated context limit (Chroma Research, "Context Rot," July 2025)
What Context Engineering Actually Means
In 2025, Andrej Karpathy defined context engineering as "the delicate art and science of filling the context window with just the right information for the next step" (Karpathy, June 2025). The idea reframes AI performance as an information-supply problem, not a wording problem - the phrasing of your prompt matters far less than what the model can actually see when it generates an answer.
Harrison Chase, CEO of LangChain, put it in more practical terms: context engineering is "building dynamic systems to provide the right information and tools in the right format such that the LLM can plausibly accomplish the task" (LangChain, June 2025). That word "dynamic" matters. A single well-crafted prompt is static. A context-engineered system pulls in the right customer record, the right prior conversation, the right document - automatically, every time, based on what the task actually requires.
By September 2025, Anthropic had formalized the discipline in its own engineering guidance, describing context engineering as curating "the optimal set of tokens during LLM inference" and naming a specific failure mode: "context rot," where models lose focus or grow confused as irrelevant information accumulates, even well inside their advertised context window (Anthropic Engineering Blog, September 2025).
Prompt engineering asks: what should I type? Context engineering asks: what does the model need to see before I ask it anything at all? For a solo founder running three tools and a handful of automations, that's the difference between an AI assistant that occasionally nails a task and one you can actually build a workflow around.
I covered the first half of this shift - why AI leverage matters for founders at all - in Why AI Is the Founder's Best Leverage Tool. Context engineering is the second half: the discipline that decides whether that leverage actually shows up in your results.
Why It's Replacing Prompt Engineering
In July 2025, Chroma Research tested 18 frontier models - including GPT-4.1, Claude 4, and Gemini 2.5 - and found every single one degraded in accuracy as the input got longer, often well before hitting its stated context limit (Chroma Research, "Context Rot," July 2025). The finding kills a comforting assumption: that a bigger context window quietly fixes reliability on its own.
Chroma ran its models through needle-in-haystack extensions, the LongMemEval benchmark, and a repeated-words task, then graded outputs with a method that matched human judgment more than 99% of the time. Across every test, the pattern held: more tokens in the window did not mean better answers. It often meant worse ones, as relevant details got buried under material the task never needed.
That's the practical case for context engineering. It isn't about cramming more into the window - it's about being deliberate over what goes in at all. A support agent doesn't need your company's entire knowledge base loaded into every call. It needs the three documents relevant to this customer's actual problem, in a format the model can act on.
I've watched teams debug an AI agent for weeks assuming the model itself was broken, when the actual problem was a retrieval step quietly stuffing the context window with outdated pricing PDFs nobody had deleted since 2023. The model wasn't the bottleneck. The information pipeline feeding it was.
If you're building or vibe-coding your own AI tools without an engineering team behind you, this is exactly where things quietly break. I walked through the broader risk-management side of that in The Vibe Coding Playbook.
The Real Reason 95% of AI Pilots Fail
In August 2025, MIT's NANDA initiative reported that 95% of enterprise generative AI pilots fail to produce measurable profit-and-loss impact, based on 150 executive interviews, 350 employee surveys, and an analysis of 300 public AI deployments representing $30-40 billion in investment (MIT NANDA, "The GenAI Divide," August 2025). The researchers traced the failure to an organizational "learning gap" - not weak models.
That gap shows up in adoption data too. McKinsey's 2025 survey of nearly 2,000 executives across 105 countries found only 23% of organizations were actively scaling an agentic AI system in any business function, while another 39% were still experimenting (McKinsey, "The State of AI in 2025").
Stack those two findings together and a pattern emerges. The 5% of pilots that work aren't running better models than everyone else. They're running better-fed models - systems where someone deliberately engineered what information reaches the AI, instead of assuming a smarter model would eventually figure it out on its own.
This is the same organizational discipline I covered from a headcount angle in The Solo Founder Playbook - context engineering is what makes that lean AI stack actually reliable enough to run a business on.
The Adoption Wave Founders Can't Ignore
In 2026, Gartner projects task-specific AI agents will show up in 40% of enterprise applications, up from under 5% in 2025 (Gartner, August 2025). That's not a gradual curve - it's an eightfold jump in a single year, and it means AI agents are about to become table stakes rather than a differentiator.
Which 40% will your business land in - the adoption wave or the cancellation pile? Gartner separately predicts more than 40% of agentic AI projects will be canceled by the end of 2027 over cost overruns, unclear ROI, or inadequate risk controls (Gartner, June 2025).
Gartner's own read is that the deciding factor isn't budget or model choice. It's whether the underlying system was engineered to hand the agent trustworthy, well-scoped context before it acts. I saw the same dynamic play out in a different AI category, which I broke down in Why Vertical AI Startups Are Outpacing Horizontal Tools - narrow, well-scoped context beats broad, general-purpose access almost every time.
Bigger Context Windows Won't Save You
In March 2026, Anthropic made a 1-million-token context window generally available for Claude Opus 4.6 and Sonnet 4.6 at standard pricing - five times the prior 200K-token standard window, enough to hold roughly 600 images or PDF pages in a single request (Anthropic, "1M Context, GA," March 2026). That sounds like the problem solves itself. The Chroma findings above say otherwise.
Cost isn't the barrier either. Epoch AI's analysis of LLM inference pricing found the price for equivalent performance has fallen roughly 50x per year on average since 2023, accelerating to closer to 200x per year in data from 2024 onward (Epoch AI, March 2025). Founders no longer have a good excuse to skip context engineering because "a bigger window is too expensive." It's cheap. The discipline of what you put in it is the actual bottleneck now.
I've had founders show me AI agent setups where every request pulls in a customer's entire history, every product doc, and every past support ticket - because "more context can't hurt." It can. It just doesn't look like it's hurting until the agent quietly gives a customer the wrong refund policy from eighteen months ago.
The Playbook: How Founders Apply Context Engineering
Notice something about every statistic above? It's enterprise data - McKinsey, Gartner, MIT. Nobody has yet published a clean survey of solo founders or five-person startups adopting context engineering specifically. That's not because it matters less at that scale - if anything it matters more, since a lean team has no data engineer quietly building these pipelines for them. It just means the playbook below is assembled from first principles, not a benchmark you can compare yourself against.
Audit what's actually in the context window. Before you touch a single prompt, log what your AI tool or agent is actually being fed on a real task - which documents, which prior messages, which tool outputs. Most founders have never looked. Most are surprised by what's in there.
Build retrieval, not one giant prompt. Instead of pasting your entire knowledge base into a system prompt, set up a retrieval step that pulls only the two or three documents relevant to the specific task at hand. Smaller, targeted context consistently beats larger, unfiltered context.
Version and test your context like code. Treat the documents, instructions, and examples you feed an AI system as a versioned asset, not a one-time setup. Change one input, re-run the same test cases, and check whether output quality actually improved.
Cut before you add. When an AI agent starts making mistakes, the instinct is to add more instructions or more background documents. Try removing irrelevant material first. Context Rot research suggests that's usually the higher-leverage fix.
Measure task success, not prompt cleverness. Stop grading your AI setup on how impressive a single response sounds. Grade it on whether the same task, run twenty times with different real inputs, succeeds consistently. That's the metric that actually correlates with ROI.
Is Your AI Stack Actually Engineered, or Just Prompted?
I work with founders auditing their AI tools and agent workflows to find where context - not the model - is the actual bottleneck. If your AI setup feels inconsistent despite a good model underneath it, let's find out why.
Book a Strategy Call →For the systems discipline that makes any of this sustainable as you scale, see The Silent Scale. And for the bigger picture on where AI fits across the Build, Move, and Grow framework, read How AI Strategy Is Redefining the Playbook.
Frequently Asked Questions
What is context engineering?
Context engineering is the discipline of deliberately curating the documents, memory, tool outputs, and prior conversation state an AI system sees before it responds, rather than focusing only on how a prompt is worded. Anthropic defines it as curating "the optimal set of tokens during LLM inference" (Anthropic, September 2025).
How is context engineering different from prompt engineering?
Prompt engineering optimizes the specific wording of a single request. Context engineering builds the dynamic system - retrieval, memory, tool outputs - that decides what information reaches the model in the first place. LangChain CEO Harrison Chase describes it as building systems that supply "the right information and tools in the right format" (LangChain, June 2025).
Why do most AI agent pilots fail to deliver ROI?
MIT's NANDA initiative found 95% of enterprise generative AI pilots fail to produce measurable profit-and-loss impact, tracing the cause to an organizational learning gap rather than weak models. Poorly engineered context - irrelevant documents, missing memory, unclear tool outputs - is a primary driver of that gap (MIT NANDA, August 2025).
Do I need a 1-million-token context window to fix my AI agent's reliability?
No. Chroma Research found accuracy degrades as input length grows, often well before models hit their stated context limits, so a bigger window alone doesn't guarantee better results. A smaller, deliberately curated context window consistently outperforms a large, unfiltered one (Chroma Research, July 2025).
Context engineering isn't a rebrand of prompt engineering with a fancier name - it's an admission that the words in your prompt were never the real lever. What the model can see, when it can see it, and how much irrelevant noise surrounds it: that's what separates the 5% of AI pilots that deliver real ROI from the 95% that quietly fade out.
You don't need a data science team to start. You need an honest audit of what your AI tools are actually being fed today, and the discipline to cut before you add. That's the whole playbook, and it works at any team size.
If you're not sure whether your AI setup has a model problem or a context problem, contact me and let's find out together.
Sources
- Andrej Karpathy, X post coining "context engineering" as the successor to prompt engineering, retrieved 2026-07-27, June 2025
- LangChain, Context Engineering for Agents (Harrison Chase's definition of context engineering as dynamic system design), retrieved 2026-07-27, June 2025
- Anthropic Engineering Blog, Effective Context Engineering for AI Agents (formal definition; "context rot" failure mode), retrieved 2026-07-27, September 29, 2025
- Chroma Research, Context Rot (18 frontier LLMs tested; accuracy degrades with input length before context limit), retrieved 2026-07-27, July 14, 2025
- Forbes, citing MIT NANDA, The GenAI Divide: State of AI in Business 2025 (95% of enterprise GenAI pilots fail to deliver P&L impact), retrieved 2026-07-27, August 22, 2025
- Gartner, Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026 (up from under 5% in 2025), retrieved 2026-07-27, August 26, 2025
- Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, retrieved 2026-07-27, June 25, 2025
- McKinsey, The State of AI in 2025: Agents, Innovation, and Transformation (1,993 respondents, 105 countries; 23% scaling, 39% experimenting), retrieved 2026-07-27, 2025
- Anthropic, 1M Context Window Generally Available for Claude Opus 4.6 and Sonnet 4.6 (5x standard window at standard pricing), retrieved 2026-07-27, March 13, 2026
- Epoch AI, LLM Inference Prices Have Fallen Rapidly But Unequally Across Tasks (roughly 50x/year average decline, accelerating post-2024), retrieved 2026-07-27, March 12, 2025
