Long context

  • Context
  • Industry term

Long context means a model can accept a huge request, such as a whole book, at once. The usable size depends on the model, the app, the input type, and room left for the answer.

Fitting and finding are separate abilities. As the input grows, each detail has more text competing for the model's attention, and a fact stated once, deep inside, is easy for it to pass over. How much this happens varies with the model and the task.

Where you’ll meet it

You upload the 300-page packet from your home loan closing and ask whether paying the loan off early would cost a fee. The AI says the documents don't mention one, though you remember signing a one-page addition about repayment somewhere in the stack.

What to do

Have it quote every passage about early repayment, with page numbers, and read those passages yourself. If the addition you remember isn't among them, find it in the packet and ask about that page on its own.

Easy to mix up

Memory loads saved facts into new conversations. With long context, whatever you paste stays in that one chat, however large it is.

Check yourself

Your company adopted a remote-work policy, and you want to know whether anything in its 150-page handbook conflicts with it. Your AI tool can send the model only passages that match a search, or the whole handbook, which fits. Which suits this question?

Go deeperBenchmarks find accuracy slipping well inside the advertised limit

In 2023, Nelson Liu and colleagues described the "lost in the middle" pattern. Answering from many documents at once, models did well with the answer near the start or end and worse in the middle. Later benchmarks such as RULER added tasks that trace a chain of facts or tally items, and scores on those fell at shorter lengths than simple lookup did.

Many systems combine long context with searching first, called “retrieval”: they search for the best matches, then send the full documents behind them. Providers can also store a long input that repeats across requests and charge less to resend it, which narrows the price gap between sending everything and sending excerpts.

A quick test of your own setup places a known fact at the beginning, middle, and end of a long input and checks that the model finds all three. Passing it sets only a floor, so a task that combines facts needs a test of its own, built the same way.

Practice Modern AI

Or take the learning quiz on a mix of every set