A map of meaning

Type "cheap flights to Lisbon" into a search engine, and it shows you a page about low-cost air travel to Portugal, although the two share hardly a word. A shop suggests a book because it is "similar" to the last one you bought, and a chatbot finds the right paragraph in a manual of three hundred pages. Behind all three lies the same idea: a model turns every text into a list of numbers, called an embedding, and these numbers mark the place of the text on a map on which texts of similar meaning lie close together. In this experiment you first draw such a map yourselves with paper cards. Then you start where a language model starts, with the numbers of single tokens, follow how they become the numbers of a whole text, and finally look at the map a real model draws from the same cards, search it and find out what it knows and what it does not.

The slides follow the experiment step by step and explain each idea at the place where you meet it.

What you need

• The 24 cards, printed on one A4 sheet and cut apart, one set per group.
• A large sheet of paper, A3 or a page of a flip chart, and a pen.
• This page: the model runs in your browser, so nothing needs to be installed. It loads about 50 MB the first time, so it is best loaded once at the start, for instance on the projector.

Work in groups of three or four.

You first: a map by hand

Lay the 24 cards on the sheet so that cards with a similar meaning lie close together and cards that have nothing in common lie far apart. There are no rules and there is no right answer, but the group has to agree on every card. When you are done, draw a line around each group of cards and give it a name.

1. Which cards were easy to place, and which did you argue about? Look in particular at cards 3 and 20, and at cards 8 and 12.
2. What made two cards close for you: the same words, the same topic, the same situation or something else?

Keep the sheet and take a photo of it, because you will compare it with the model's map later.

Step 1: from a place to numbers

Draw two lines across the middle of your sheet, one from left to right and one from bottom to top, as on a chart, and mark both from −10 to 10. Now every card has a place that you can write down as two numbers: how far to the right it lies and how far up.

3. Write down the two numbers for four cards of your choice.

4. Give another group only the two numbers of one card, not the card itself. Can they find the card on your sheet? Can they find it on their own?
5. Find two cards that you had to put close together although they differ in something important. What would you need to keep them apart?

This is all an embedding is: a list of numbers that says where something lies on a map of meaning. Your map has two numbers per card because a sheet of paper has two directions. The model you meet next uses 384 numbers for every token and every text, so its map has 384 directions, which nobody can draw, but the idea is the same: similar meaning, close places.

Step 2: every token gets numbers

In Next-token prediction you saw that a language model does not read words but tokens, and that the first thing it does is replace every token by a list of numbers. These lists are embeddings, and they are where the map begins. The model below is small and it is not a chatbot, but it starts in the same way, with a table that holds a list of 384 numbers for each of the 30,522 tokens it knows. Load it and compare two words. Under the similarity you see the tokens the model reads. The first and the last are not words but markers that the model puts around every text, [CLS] (short for classification) at the start and [SEP] (short for separator) at the end. They get numbers of their own and count in the average of Step 3 like every other token. A single word is mostly one token, so the numbers you compare are essentially those of that token, after the processing that Step 3 describes.

The similarity of two words is one number between −1 and 1. Think of the 384 numbers as an arrow from the centre of the map: 1 means that two arrows point in exactly the same direction, 0 that they have nothing to do with each other, and −1 that they point in opposite directions. This model hardly ever goes below 0: I love this film. and Please restart your router., two texts with nothing in common, get −0.06. Single words that have nothing in common still reach about 0.2 to 0.3, puppy and invoice for instance 0.22, while for whole texts values from about 0.3 already mean that they are clearly related. What counts is therefore the order of the values, not the value itself.

6. Compare dog with puppy, with cat and with car. Put the three in order before you press the button.
7. Compare hot with cold and with warm, and warm with cold. Is the opposite far away?
8. Find the pair of different words with the highest similarity you can, and a pair of words that look alike but get a low one. Then try to find a pair of words with a negative similarity.

Arithmetic with meaning

Because the numbers are directions, you can compute with them. Type three words below, king, man and woman for a start: the model turns each of them into its 384 numbers, the calculation subtracts the numbers of man from those of king and adds those of woman, position by position, and the list shows the words whose numbers are most similar to the result. A click on a sign turns a minus into a plus and back. The result is searched among a fixed list of about 400 English words, leaving out the words you typed.

9. Compute paris − france + germany, puppy − dog + cat and walked − walk + swim. Which direction does the subtraction remove in each case, and what does the addition put in?
10. Try the way back, queen − woman + man, and find a calculation that fails. How close do the values of the first words come to each other when it fails?

The same in R

The table word_vectors holds the 384 numbers the same model gives 48 of these words, one row per number. This box computes king − man + woman and lists the words whose numbers are most similar to the result, leaving out the three words it started with. How the similarity is computed from the numbers you will see in Step 7.

word <- function(w) word_vectors |> filter(word == w) |> pull(value)
target <- word("king") - word("man") + word("woman")

word_vectors |>
  filter(!word %in% c("king", "man", "woman")) |>
  group_by(word) |>
  summarize(similarity = sum(value * target)) |>
  arrange(desc(similarity))

Step 3: from tokens to a text

A word has its list of numbers, but a sentence is more than a list of words. The drawing shows, in a simplified way, how the model gets from the numbers of its tokens to the numbers of a whole text.

• Cutting: the tokenizer cuts the text into tokens and adds [CLS] and [SEP].
• Looking up: every token gets its list of 384 numbers from the table, the same entry wherever the token stands, together with a few numbers that mark its position in the text.
• Adjusting: in six rounds, every token looks at all the others and changes its numbers towards those that matter for it. This is the attention of Next-token prediction, and it is why "bank" ends up with different numbers after "river" than after "interest rates".
• Averaging: the model takes the average of all lists, position by position, and scales the result so that its arrow has the length 1. This one list of 384 numbers is the embedding of the text.

Measured with this model, the token "bank" in "The bank raised its interest rates." and in "We watched the ducks from the bank of the river." starts from the same entry in the table and ends with a similarity of 0.76, while in two sentences about money it ends at 0.88. The same word gets different numbers in different sentences, which is what makes these embeddings more than a dictionary.

11. Compare unbelievably with incredibly in the demonstrator of Step 2, and look at the tokens. What do you notice, and how could the cutting explain the similarity?
12. Look at the tokens of Python and of ChatGPT, and of the German sentence Ich bin krank. What does the model make of a word it has not seen often?
13. Compare bank with money and with river. Why can a single word not decide this, while a sentence can?

Where the numbers come from

Nobody chose the 384 directions of this model; they were learned in two stages. In the first, a model reads huge amounts of text, in which single words are hidden, and has to guess them from their neighbours. To guess well, it has to give similar numbers to words that can stand in the same places: "puppy" where "dog" could stand, "cold" where "hot" could stand. This is the old idea behind every embedding, in the words of the linguist John Rupert Firth from 1957: "you shall know a word by the company it keeps". It is not about words that stand next to each other, since dog and cat rarely do, but about words that have the same neighbours, which is also why hot and cold came out close in Step 2. The model on this page learned this stage from a larger model that had been trained in this way.

Words in similar places are not yet texts with similar meaning, so the second stage trained on whole texts: more than a billion pairs of texts that belong together, such as a question and its answer, two questions that ask the same thing, or the title and the abstract of a paper. Each time, the numbers of the two texts of a pair were moved closer together and those of texts picked at random further apart, until a search by meaning worked. What a single one of the 384 directions means is written nowhere, and most of them mean nothing a person could name.

Step 4: the model's map

Now the model reads the 24 cards, the way the drawing shows, and turns each of them into 384 numbers, which takes a few seconds; if you loaded it in Step 2, it is already there. Because 384 directions cannot be drawn, the map shows their shadow on two: the two directions in which the cards differ most, a method called principal component analysis. Point at a dot to read its card.

14. Compare the model's map with your sheet. Which of your groups did the model form as well, and where does it disagree with you?

15. Look at the two directions of your own sheet. Would the model have chosen them, and could you name two of its directions on its map?

Step 5: search by meaning

Type a query into the search next to the map. The model turns the query into 384 numbers as well, and the search lists the cards whose numbers are most similar to those of the query; on the map the query appears as a star. Below the list you see what a classic search for the same words would have found. The values are similarities as in Step 2, now between the query and each card.

16. Search for I can't get home tonight. Which card comes first, and what would a keyword search have found?
17. For three groups on your sheet, find a query that brings one of their cards to the top without using any word of that card. Then compete with another group: who finds the query for which a keyword search and the search by meaning disagree most?
18. Search for money, river and snake. Does the model keep the two bank cards and the two python cards apart?
19. Search for a pet that purrs and for Ich bin krank. What did you expect, and what happened?

Step 6: what the map does not know

In the demonstrator of Step 4, below the map and the search, you can compare two texts of your own and get their similarity.

20. Compare I love this film. with I do not love this film., and then with What a wonderful movie!. Which pair is more similar, and does it surprise you?
21. Compare The dog bit the man. with The man bit the dog., and Vaccines cause autism. with Vaccines do not cause autism..
22. Find two texts that mean the same and stay below 0.3, and two texts that mean the opposite and get above 0.8.

Step 7: inside the numbers

In R you can look at the 384 numbers of every card yourself, computed by the same model as on the map, and see that the search is nothing but one sum.

The cards

The table cards holds the 24 cards with the numbers printed on them:

cards

A card as numbers

The table vectors holds the 384 numbers of every card, one row per number, with the card in card, the position in dim and the number in value. These are the first ten numbers of card 1:

vectors |>
  filter(card == 1) |>
  head(10)

23. Can you see from these numbers that card 1 is about a train?

Similarity is one sum

To get the similarity of two cards, multiply their numbers position by position and add up the 384 products. This model scales the numbers of every text so that its arrow has the length 1, and then this sum is exactly the similarity from the search. Here for cards 1 and 7:

one <- vectors |> filter(card == 1)
seven <- vectors |> filter(card == 7)
sum(one$value * seven$value)

24. Compute the similarity of card 1 with card 19 and with card 5. What do the three values tell you?

The nearest cards

The same sum for all cards at once is a search engine by meaning. This box puts the numbers of card 3 next to those of every card, adds up the products for each card and sorts the result:

three <- vectors |>
  filter(card == 3) |>
  select(dim, value_3 = value)

vectors |>
  left_join(three, by = "dim") |>
  group_by(card) |>
  summarize(similarity = sum(value * value_3)) |>
  arrange(desc(similarity)) |>
  left_join(cards, by = "card")

25. Which cards are nearest to card 3, and which are nearest to card 20? Change the box to find out.

What real systems do with it

A search engine, a shop or the help desk of a university does the same as the last box, with millions of texts instead of 24 and with special databases that find the nearest numbers without adding up every sum. Chatbots that answer from your documents turn your question into numbers first, look up the most similar paragraphs and put them into the context of the language model, which then writes its answer from them; this is called retrieval-augmented generation. Recommendations, the detection of duplicate questions and the sorting of feedback into topics all work with the same maps. And inside every language model, the first thing that happens to each token is that it is replaced by a list of numbers like these, with everything after that working on numbers, as Next-token prediction describes and as you saw in Steps 2 and 3.

Extensions

1. Your sheet against the model: measure with a ruler which three cards lie nearest to card 1 on your sheet, and compare them with the three nearest cards in R.
2. Make the model fail: find a query for which the first card is clearly wrong, and explain what the model latched onto instead.
3. A question for a chatbot: ask a chatbot something about a long document you give it, and ask it afterwards which paragraphs it used. Would a search by meaning have found the same paragraphs?

Discussion

Write down your own answers first, then discuss them in your group.

1. Your map and the model's map differ. Which of the two is right, and what would it even mean for a map of meaning to be right?
2. A university builds a chatbot that answers questions by searching its regulations by meaning. After Step 4, what could go wrong, and how could you notice it?
3. A student says: "The model understands the cards, otherwise it would never have found card 1 for I can't get home tonight." How would you answer, based on what you saw in this experiment?