When you talk to an AI model, there is no persistent memory between sessions. Every time you open a new conversation, the model starts from scratch. But within that session, it can see and process everything you have written, everything it has replied, and any document you have pasted in. That active workspace is called the context window.

For years it was one of the most frustrating limits of language models. Today it is one of their most powerful assets.

What is the context window

The context window is the amount of text a model can see at any given moment while generating a response. It is not a saved file or a retrievable history: it is literally what is loaded into the model’s active memory at the moment it processes your request.

That space includes everything: the system message with initial instructions, the conversation history, any document you have attached, and the response it is generating. When something falls outside that space, the model cannot access it.

The unit of measurement is the token, which corresponds roughly to three-quarters of a word in English. A model with a 200,000-token window can process around 150,000 words in a single session — the equivalent of a long novel.

How it is measured and what that means

In the early years of modern LLMs, context windows were small: 4,096 tokens, then 8,000, then 32,000. Those limits forced users to split documents, summarise prior conversations, or work in fragments.

Today the most capable models work with windows of 128,000 to 1,000,000 tokens. That transforms the possibilities:

  • You can paste the full contract, not just the paragraph that concerns you.
  • You can hold a long conversation without losing the thread.
  • You can load several related documents and ask the model to analyse them together.
  • You can work on a long project without having to summarise context every few responses.

However, size is not the only factor that matters. A model’s quality of attention varies depending on where content sits in the window: models tend to pay more attention to the beginning and end of a long text than to the middle. This is the so-called lost-in-the-middle effect, documented in several benchmarking studies.

What happens when it fills up

When a conversation exceeds the context window, something has to go. Depending on the platform, this can mean:

The model starts forgetting the older parts of the conversation. Some systems slide the window forward, dropping the oldest messages as new ones are added.

The platform warns you and offers to summarise the history or start a new session. This is the most transparent behaviour.

The model makes coherence errors without warning, because it no longer has access to information you provided earlier. This is more dangerous because it can go unnoticed.

If you work on long projects with AI, learning to detect when context is running low is a concrete practical skill.

How to use it well in practice

Understanding the context window changes how you structure your work with AI. Some practices that make a real difference:

Put critical instructions at the beginning and repeat them at the end in long sessions. The model pays more attention to those positions. Instructions buried in the middle of a very long context are applied less reliably.

Load complete documents when you can. A model with 200,000 tokens can analyse an 80-page report in one go. There is no need to split it: the result is more coherent when the model sees the whole thing.

Monitor session length in tasks that require coherence. If you are working on something that requires the model to remember early details, consider making an explicit summary of the project state before the window fills.

Distinguish between session memory and persistent memory. Some systems offer memory between sessions through external tools: vector databases, summary files, RAG systems. That is not the context window — it is an additional architectural layer that feeds relevant information into the window when needed.

The memory that matters

The context window is not the only form of memory in an AI system, but it is the most immediate. It is the space where reasoning happens, where pieces connect, where the model can follow the thread of what you are building.

Working well with it does not require expertise in model architecture. It requires understanding one simple idea: the model knows nothing that is not inside that window at that moment. What you put there, how you structure it, and how much space you leave for reasoning matters as much as the question you ask.

In the coming years, context windows will continue to grow. But the skill of structuring information so a model can use it well does not become obsolete with each new release — if anything, it becomes more valuable.