Context compaction
- Context
- Industry term
Context compaction makes room in a long AI conversation by replacing earlier messages with a shorter record of the conversation. Some details may be lost, though the conversation can continue.
The model, the trained AI that generates replies, works with the information the app sends it. After compaction, the app sends the shorter record instead of the messages it replaced. A missing detail must be sent again before the model can use it, even if the original message is still visible on screen.
Where you’ll meet it
Partway through drafting a client proposal, your AI app reports that it has compacted the conversation. Its next draft includes a service you ruled out near the start because it would exceed the client's budget.
What to do
Restate the budget and the excluded service before asking for another draft. Then save those decisions in the project's instructions or a file the app rereads after compaction, so the next summary isn't their only record.
Easy to mix up
| Context compaction | Context rot |
|---|---|
| The original messages were replaced by a shorter record | The information is still supplied, but the model uses it less reliably |
Check yourself
A long chat is compacted, then later compacted again. The first summary leaves out a supplier's delivery date. You never supply the date again. Which explanation better fits its absence from the second summary?
Go deeperShorter records take different forms and can interrupt reuse of earlier work
Anthropic's September 2025 context-engineering article describes model-written summaries. OpenAI's compaction documentation describes a different format: an encrypted record, encoded so people can't read it directly. It preserves information from earlier steps in fewer tokens, the small units models use to process text.
OpenAI's prompt-caching documentation describes another tradeoff. Caching can reuse calculations for text the model has already processed at the start of a request. Replacing early messages changes that text, so earlier calculations may no longer match. Anthropic's threshold-compaction documentation also notes that generating a summary adds another request to the model, which counts toward usage.
Sources
- Compaction overview, Anthropic Claude Docs
- Compaction, OpenAI API docs
- Effective context engineering for AI agents, Anthropic Engineering