A Million-Token Memory That Still Forgets Page 600
Thirteen AI models now advertise context windows of a million tokens or more, some up to 10 million. On every model benchmarked, the length where quality actually holds is shorter than the advertised figure.
By the UISC BD Editorial Desk · United Information Service Center · Published 13 September 2026 · 6-minute read
You start a conversation with an AI tool and it is sharp. Two hours and forty messages later it contradicts itself, forgets an instruction you gave at the start, and seems to have lost the thread.
It is not tired. It has run into the limits of its context window.
What a Context Window Is
A model has no memory between requests. Everything it knows about your conversation is the text sent to it each time: your instructions, the documents you shared, and the whole chat so far.
The context window is the maximum amount of that text, measured in tokens, a model can take in at once. Once a conversation exceeds it, something has to be dropped or summarised.
How Big They Have Become
In 2026, context windows range from about 128,000 tokens on standard models to 10 million on models such as Llama 4 Scout and Gemini 3 Pro. Thirteen models now ship windows of 1 million tokens or more — enough to hold several long books at once.
That sounds like the problem is solved. It is not.
Advertised Versus Effective
The effective context — the length at which quality actually holds — falls short of the advertised maximum on every model ever benchmarked.
The illustration used by one 2026 comparison is worth quoting in substance: load a 900,000-token document into a million-token model and ask about page 600, and the model may confidently give a wrong answer. The information is present. Its attention has drifted.
Lost in the Middle
The failure is not random. Models recall material at the beginning and end of a long context more reliably than material in the middle.
That has direct practical consequences:
- Put the most important instruction at the start, and repeat it near the end of a long prompt.
- Put the question after the document, not before it.
- Do not bury a critical fact in the middle of a 200-page paste and assume it will be found.
Why Bigger Is Not Always Better
Cost. Filling a 1-million-token window costs about $0.14 on DeepSeek V4 Flash and about $10.00 on Claude Fable 5 — a 71-times spread. Architectural techniques such as sparse attention, ring attention and efficient KV-cache management make huge windows possible, but they do not make them free.
Speed. More input takes longer to process before the first word of an answer appears.
Accuracy. As above, more is not the same as better remembered.
What to Do Instead
Start new conversations more often. When a thread drifts, open a fresh one with a short summary of what matters. This is the single most effective habit.
Send the relevant part, not everything. Retrieval — pulling only the passages that matter — usually beats stuffing a whole archive into the window. That is the approach behind RAG.
Keep standing instructions short. A three-page system prompt spends your context and your budget on every single message.
Verify answers from long documents. Treat them with the care described in our checking routine.
The Mental Model to Keep
A context window is a desk, not a filing cabinet. A bigger desk holds more paper, but the more you pile on, the harder it is to find the page you need. The skill is keeping the right papers on the desk.
Related reading
- What a Token Is, and Why AI Prices Fell About 80% in a Year
- MCP Explained: The Protocol That Connects AI to Your Tools
- How to Run AI on Your Own Computer: Ollama and llama.cpp
- How Well Does AI Understand Bangla? A Practical Guide
Sources
- "AI model context window comparison 2026: advertised vs. real," elvex — elvex.com
- "LLM context window 2026: 128K to 10M tokens — which to use," TokenMix — tokenmix.ai
- "LLM context windows explained: 4K to 1M tokens (2026)," DevTk.AI — devtk.ai
- "LLMs with largest context windows," Codingscape — codingscape.com
