Several readers pointed out that I launched my Context Engineering series without first explaining where the term came from or how it evolved from Prompt Engineering.
They raised a fair point. So I’m taking a step back to explore those origins, explain the shift, and set the foundation for the rest of the series.
Consider this article the context behind Context Engineering. 🙂
Background
In mid-2025, a quiet reframing swept through the teams building serious AI systems. The skill that everyone had been calling prompt engineering turned out to be only a small part of the real work. The larger discipline, the one that determines whether an AI agent is reliable enough to place in front of a customer or a regulator, now has a name: context engineering. This article explains what it is, why it emerged, the framework practitioners use to apply it, and what it means for financial institutions deploying generative and agentic AI.
Where the term came from
The phrase gained traction in June 2025. Andrej Karpathy gave it its clearest articulation, arguing that the label prompt engineering understated the actual work of building agents. Around the same time, Shopify chief executive Tobi Lutke amplified the same idea. Their shared point was simple. Prompt engineering suggests crafting a single instruction. Context engineering describes the broader discipline of filling the model context window with the right information: instructions, retrieved knowledge, memory, tool descriptions, and prior outputs, all structured so the model can use them effectively.
Karpathy offered a mental model that has proven durable. The large language model behaves like a new kind of operating system. The model is the CPU, and its context window is the RAM, serving as working memory with strictly limited capacity. Context engineering is the job of deciding what gets loaded into that working memory at each step.
The working definition
The framing published by Anthropic in its engineering guidance in September 2025, alongside Claude Sonnet 4.5, is the definition most practitioners now anchor to. Because large language models operate under a finite attention budget, good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of a desired outcome. The central question it poses is direct: what configuration of context is most likely to generate the model desired behavior?
That phrase, the smallest possible set of high-signal tokens, is the heart of the discipline. More context is not better. The work is curation, not accumulation.
Prompt engineering versus context engineering
Prompt engineering was the primary focus in the early era of LLM applications. It centered on writing and organizing instructions, and it worked well for one-shot tasks such as classification and simple text generation. Context engineering is the natural progression. It encompasses the strategies for curating and maintaining the optimal set of tokens during inference, including everything that lands in the window outside the prompt itself.
The distinction is one of scope. Prompt engineering is about a sentence. Context engineering is about the entire pipeline that produces that sentence and everything surrounding it. Prompt engineering is not obsolete. It is now one layer inside a larger system.
The four pillars
The definition of context engineering varies across sources, with some frameworks listing four elements and others five or six. The framing with the most staying power is the one popularized by Lance Martin at LangChain: Write, Select, Compress, and Isolate.These are general principles that apply to any agent system, whether built on LangGraph, the OpenAI Agents SDK, Anthropic Claude tooling, or a raw API loop.
1- Write
It means saving context outside the window so that it survives across turns, using scratchpads, memory files, or task plans. This prevents the model from re-deriving the same facts repeatedly. A code-review agent, for example, logs issues and fixes for each file rather than rescanning the entire codebase, building a persistent and queryable memory over time.
2- Select
Itmeans pulling the right information into the window at the right moment, through retrieval, file reads, or tool calls. This is where similarity search, embedding-based retrieval, and semantic tool selection operate, and it is the home of retrieval-augmented generation.
3- Compress
It means reducing what is already in the window down to the tokens that still matter. In practice this covers conversation summarization, tool-output compression, and state-based compression that carries forward only a running summary.
4- Isolate
It means splitting work across separate context windows so that sub-tasks do not contaminate one another. This is the basis for multi-agent designs, sandboxed execution environments, and structured state that keeps unrelated information apart.
Context rot
The concept that gives context engineering its urgency is context rot, and it is valuable precisely because it is measured rather than asserted. Context rot is the observable degradation in model output quality as input length grows, and it sets in well before the context window is full.
In July 2025, Chroma Research tested eighteen frontier models and found that every one of them degraded as context length increased. A model advertising a two hundred thousand token window can show serious accuracy loss at fifty thousand tokens. The practical lesson is blunt. A full window is not a used window. Loading everything available does not help the model. It actively degrades performance.
How RAG fits
Retrieval-augmented generation and context engineering are often conflated, so the relationship is worth stating plainly. RAG is a technique that implements one specific part of context engineering, namely the Select pillar. Context engineering is the broader discipline that governs the entire context window, of which retrieved documents are only one component. An institution that has invested heavily in RAG has addressed one pillar of four.
Why it matters for financial services
For financial institutions, context engineering is where accuracy, cost, and auditability are won or lost in production. Several of its principles map directly onto the concerns of risk, compliance, and model-validation functions.
The principle of the smallest high-signal token set maps to reducing the hallucination surface, which is the outcome that risk and validation teams care about most.
Isolation maps to data segregation and least-privilege access, because keeping an agent context bounded is a control on what data and systems it can touch.
Compression and memory management map to cost control at scale, where uncontrolled context growth translates directly into token spend.
There is also a security dimension. Keeping retrieved or untrusted content isolated from instructions is a defense against prompt injection, which matters for any customer-facing or regulated deployment.
Framed this way, context engineering is not only an engineering concern. It is a governance concern, and it belongs in the conversation whenever an institution moves an AI agent from demonstration to production.
Takeaway
The shift from prompt engineering to context engineering reflects a maturing field. The question is no longer how to phrase a single instruction well. It is how to assemble, structure, and maintain the complete set of information an agent needs to perform reliably, at acceptable cost, and within controls that a regulator would accept. For leaders deploying generative and agentic AI in financial services, that is the discipline worth building capability around.
Explore more: Start Here · Topics · Archive · About