Context Engineering
Context Engineering
Context Engineering is the discipline of curating and maintaining the optimal set of information (tokens) fed into a Large Language Model (LLM). While prompt engineering focuses on how to phrase a single question, context engineering builds dynamic systems to provide the exact instructions, facts, and tools an AI agent needs to reliably perform multi-step tasks.
Transitioning from prompt optimization to context engineering allows you to manage the entire information ecosystem surrounding an AI agent. This approach relies on four core strategies to maximize the model's limited working memory:
- Context Selection: Using Retrieval-Augmented Generation (RAG) and semantic searches over vector databases or knowledge graphs to pull the most relevant facts or documents.
- Context Compaction: Condensing or summarizing long documents and past conversation histories to save token space and reduce latency
- Context Writing: Allowing the model to document its own observations, save state variables, and write notes to long-term memory for use in later steps
- Context Isolating / Filtering: Providing only the specific set of tools and data needed for the immediate next step, which prevents context confusion.
Context Engineering is the next phase, where you architect the full context, which in many cases requires going beyond simple prompting and into more rigorous methods to obtain, enhance, and optimize knowledge for the system.
From a developer's point of view, context engineering involves an iterative process to optimize instructions and the context you provide an LLM to achieve a desired result. This includes having formal processes (e.g., eval pipelines) to measure whether your tactics are working.
Given the fast evolution of the AI field, I suggest a broader definition of context engineering: the process of designing and optimizing instructions and relevant context for the LLMs and advanced AI models to perform their tasks effectively. This encompasses not only text-based LLMs but also optimizing context for multimodal models, which are becoming more widespread. This can include all the prompt engineering efforts and the related processes such as:
- Designing and managing prompt chains (when applicable)
- Tuning instructions/system prompts
- Managing dynamic elements of the prompt (e.g., user inputs, date/time, etc.)
- Searching and preparing relevant knowledge (i.e., RAG)
- Query augmentation
- Tool definitions and instructions (in the case of agentic systems)
- Preparing and optimizing few-shot demonstrations\
- Structuring inputs and outputs (e.g., delimiters, JSON schema)
- Short-term memory (i.e., managing state/historical context) and long-term memory (e.g., retrieving relevant knowledge from a vector store)
- And the many other tricks that are useful to optimize the LLM system prompt to achieve the desired tasks.
In other words, what you are trying to achieve in context engineering is optimizing the information you are providing in the context window of the LLM. This also means filtering out noisy information, which is a science on its own, as it requires systematically measuring the performance of the LLM.
Anthropic Context Engineering
Context is a critical but finite resource for AI agents. In this post, we explore strategies for effectively curating and managing the context that powers them
After a few years of prompt engineering being the focus of attention in applied AI, a new term has come to prominence: context engineering. Building with language models is becoming less about finding the right words and phrases for your prompts, and more about answering the broader question of “what configuration of context is most likely to generate our model’s desired behavior?"
Context refers to the set of tokens included when sampling from a large-language model (LLM). The engineering problem at hand is optimizing the utility of those tokens against the inherent constraints of LLMs in order to consistently achieve a desired outcome. Effectively wrangling LLMs often requires thinking in context — in other words: considering the holistic state available to the LLM at any given time and what potential behaviors that state might yield.
In this post,Anthropic will explore the emerging art of context engineering and offer a refined mental model for building steerable, effective agents.
Anthropic https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
