Context Engineering: Your Window Is Smaller Than It Says

Of all the agent-engineering terms 2026 produced, context engineering is the one with the least hype and the most measurement behind it. The measurements are humbling. Long-context benchmarks that require anything beyond literal string matching show frontier models dropping from near-perfect to under seventy percent accuracy at a mere 32K tokens. Independent needle-variant studies find models effectively use ten to twenty percent of their advertised window. And accuracy starts sliding thirty to fifty percent before the stated limit — on code released for anyone to reproduce.

Budget for the window you get

The practical stance: treat the advertised number as the tank size and ten to twenty percent as the fuel gauge you plan around. That reframes context from a bucket you fill into a budget you spend. Every retrieved document, every tool schema, every transcript line competes for the same scarce attention — and past a point, adding context makes answers worse, not safer.

Position is part of the budget. Attention over long inputs follows a U-curve: material at the start and end is recalled well, the middle is a valley — and the penalty compounds when an answer requires hopping between several middle-of-context facts. The working rule is unglamorous: instructions at the start and restated at the end, load-bearing facts near the edges, bulk reference in the middle where losing it hurts least.

Compaction without a store is deletion

Agents outlive their windows, so something must shrink history. The documented failure mode is doing this with naive summarization: three separate 2026 studies catalog runs where the summary silently dropped the one fact a later step needed — and once the raw context is gone, that fact is not degraded, it is destroyed.

The pattern that holds up pairs compression with an addressable store: compact aggressively, but keep every evicted span retrievable by pointer, so the summary is an index rather than a replacement. The same discipline applies to long-term memory: one benchmark found that even when stored facts carried valid invalidation metadata, retrieval served the stale version 60 percent of the time. Writing metadata is not the same as ranking by it — verify the read path, not just the write path.

2026: compaction becomes a training problem

Two shifts this year sharpened the field. First, the framing: Karpathy defined context engineering as "the delicate art and science of filling the context window with just the right information for the next step" — and Anthropic gave the failure a name, context rot: irrelevant tool outputs, outdated intermediate state, and redundant re-reads accumulating until the model silently attends less accurately to the signal buried under the noise. Rot is not overflow; a window can be half empty and already rotten.

Second, the response: compaction is moving from a serving trick into the training loop. CompactionRL trains agents with compaction in the loop, so the model learns to act on compacted histories rather than being surprised by them at inference — and compaction-aware training beats post-hoc summarization of a model that never saw a summary during training. Related serving work found summarization output largely input-invariant, which makes block-parallel compaction practical: operators control summary volume by block count instead of praying over one long summarization call. The direction of travel is clear: the summary is becoming part of the agent, not a patch applied to it.

The same problem, seen twice

Context-window management and memory retrieval are one problem viewed from two sides: what does the model see right now, and can it get back what it currently does not? Engineer the first with budgets and position, the second with addressable stores and verified recall — and be suspicious of any design that answers either question with "the window is big enough."