Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Context window

  • Context
  • Standard term

A context window is how much text an AI model can take in at once, measured in tokens (small chunks of words). Your messages, files, and instructions for that turn all have to fit.

A model answers from what one request contains, so each time you send a message, the app resends the conversation so far, and the reply has to fit in the room left. A long chat grows toward the limit, where the app trims or summarizes older messages, or refuses.

Where you’ll meet it

You paste a full year of meeting transcripts into a chat and ask for every decision about the budget. The app refuses, saying the message is too long for this model.

What to do

Split the job so each piece fits. Paste one quarter's transcripts into its own chat and ask for the budget decisions with their dates. Then open a fresh chat, paste in the four short lists, and ask for the combined answer.

Easy to mix up

Memory is saved notes about you that the app carries into new chats. The context window holds only the current chat, and each new chat gets a window of its own.

Check yourself

You ask your coding agent why your app's sign-in page breaks. A few minutes and two short messages later, it says it has to compress the conversation to keep going. What most likely filled the window?

Go deeperHidden text takes room, and models use a long input's middle less

Some of a window holds text the user never sees, such as the app's own instructions and a description of each tool it can call. Models that reason before answering spend window on that reasoning too. In English a token averages about three-quarters of a word, and other languages and code often take more tokens for the same content.

Nelson Liu and colleagues found that models use information at the start and end of a long input more reliably than information in the middle, a pattern they called “lost in the middle.” The advertised size says what fits, and placement still decides how well each part gets used.

Some products search your documents and send only the passages that match the question. Many providers also keep the processed form of a request's unchanged opening, such as long standing instructions, so resending it is cheaper and faster, though it fills the same room. When a workflow built on a model misbehaves, look at the full request as sent and count each part's tokens.

Practice Modern AI

Or take the learning quiz on a mix of every set