Every word is a place
A model cannot work with letters, so each word becomes a position — a list of numbers describing where it sits relative to everything else. Words used in similar ways end up as neighbours. Nobody arranged that. It fell out of reading.
6 min · Beginner · Explorable
Similar things end up near each other
Imagine a huge map where you place every word you know. Words that show up in the same kinds of sentences go close together — “dog” near “cat”, “rain” near “storm”. Do that carefully enough and the map starts to encode meaning, because in language, meaning largely is the company a word keeps.
Not a dictionary
The model has no definitions. It only knows which words appear near which others, across billions of sentences.
Not two dimensions
The real map has hundreds of directions at once. The one below is flattened so you can actually look at it.
Not designed
Nobody drew these neighbourhoods. They are what you get when you predict text well enough for long enough.
Explore it, then run the analogy machine
Tap any word to see its nearest neighbours. Then press one of the analogy buttons — the gold arrow and the pink arrow are the same length and point the same way, which is the entire trick. “The direction from a country to its capital” is a real thing on this map, and it is the same direction everywhere.
Nearest to “king”
Analogy machine
Neighbourhoods
Two dimensions, arranged by hand so the arithmetic lands exactly. A real embedding space has hundreds of dimensions and arrives at these relationships on its own, from nothing but text — which is the genuinely surprising part.
In plain words
When you subtract one word from another you are asking “what is the direction between these two?” Add that direction to a third word and you arrive somewhere new. That is all an analogy is here — arithmetic on positions.
Now find bank
Type “bank” into the search box, or tap the word with the dashed ring. It is stranded — halfway between the money neighbourhood and the river one, genuinely close to neither. A single position cannot hold two meanings at once.
This is why context matters
Modern models do not give a word one fixed position. They compute it fresh each time, from the surrounding words — so “bank” in a sentence about loans lands somewhere different from “bank” in a sentence about fishing.
This is what attention is for
Deciding which neighbouring words should pull “bank” toward money and which toward water is exactly the job attention does. The map is the starting point; attention moves things around on it.
What this explains
Why search finds things you did not type
Semantic search compares positions, not spellings. A document about “annual leave” answers a question about “time off” because those sit near each other, even with no shared words.
Why models absorb bias
If text consistently places one group of words near another, the map records it — including associations nobody would defend. The map is a mirror of what was written, not a judgement about what is true.
Why recommendations work
The same trick applies to products, songs and articles. Place them by how people use them, and “similar to this” becomes “close to this on the map”.
Why rare words behave badly
A word that appeared a handful of times has been placed with almost no evidence. It sits in roughly the right area but not precisely — which is why models are unreliable on unusual names and niche jargon.
Next
The map is stage two of four
Tokens become positions, positions get rearranged by attention, and a prediction comes out. The next lab runs all four stages on a sentence you write.