Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Embeddings

  • Context
  • Standard term
2 steps

An embedding is a list of numbers an AI model produces for a piece of text, an image, or audio, so a computer can compare items. Items the model treats as related get nearby numbers.

The model learns from many matched pairs, such as a question and the passage that answers it, or a photo and its caption. Training adjusts it until each pair lands close and unmatched items land apart. Comparing two embeddings is quick arithmetic, and an index built ahead of time finds close matches without checking every item.

Where you’ll meet it

Your team's help desk tool offers to merge two new tickets as duplicates: “Order 4471 arrived damaged” and “Order 4417 showed up broken.” They share almost no words. The tool compared the two tickets' embeddings and found them nearly the same.

What to do

Compare the order numbers before you accept the merge. An embedding records what a ticket is about, a broken delivery, and barely registers which order it names, so complaints about two different orders land side by side. Merged, one customer's ticket would close without an answer.

Easy to mix up

Semantic search is what you use: a search box that ranks results by meaning. Embeddings are the numbers underneath it, which only software reads.

Check yourself

A shopping site's “similar items” feature switches to a newer embedding model for new products and keeps the stored embeddings of older products as they were. What happens to suggestions that compare new items with old ones?

Go deeperPick an embedding model by testing it on your own documents

Representing words as points took off with word2vec, which Tomas Mikolov and colleagues at Google published in 2013. It gave each word a point in space, and the arithmetic king minus man plus woman landed near queen. Current models embed whole passages, images, and code, often as hundreds or thousands of numbers each.

Closeness reflects whatever the training pairs taught, so a model trained mostly on general web text can treat two drug names or two contract clauses as near twins when a specialist would not. Systems that need exact matches often run keyword search alongside embeddings.

Public leaderboards such as MTEB, the Massive Text Embedding Benchmark, average many tasks and can mislead for one field. A small set of your own queries, with the right documents marked, shows which model ranks your material best. OpenAI, Cohere, and open models such as BGE are common choices.

Practice Modern AI

Or take the learning quiz on a mix of every set