Embeddings · Measuring similarity

The real measure

Distance is the picture, angle is the maths

An embedding turns each word into a position in space, and words used alike end up near one another — which is what a map like this shows. But real embeddings aren’t compared by the gap between two dots. They’re compared by the angle between them, measured from the centre. It’s called cosine similarity: 1 means pointing the same way, 0 is a right angle, negative is opposite.