Context Window

Definitions

1. The maximum amount of text (measured in tokens) that a language model can process in a single interaction. Everything the model can 'see' and 'remember' during a conversation. Once you exceed the context window, earlier information is effectively forgotten.

   In plain English: How much text an AI can read and remember at once — like the size of its working memory. Older messages get forgotten when the window fills up.

2. Context window size has grown dramatically: GPT-3 had 4K tokens, GPT-4 expanded to 128K, and models like Claude now support 200K+ tokens. But longer context doesn't mean perfect recall — models tend to lose information in the middle of very long contexts (the 'lost in the middle' problem).

   Example: 'We stuffed the entire codebase into the 200K context window, but the model still missed the bug. Turns out it was in the middle 50K tokens where attention drops off.'

   Source: practical limitations

Etymology

2017

The Transformer architecture introduces fixed-length attention, creating the concept of a 'context window' — how much text the model can consider at once

2023

Context windows expand from GPT-3's 4K tokens to Claude's 100K and GPT-4 Turbo's 128K, changing what LLMs can process

2024

Million-token context windows (Gemini 1.5) challenge the need for RAG in some use cases, though cost remains a factor

Related Terms

• Large Language Model
• Transformer
• Prompt Engineering
• GPT
• RAG
• Tokenizer
• Context Window