Embeddings · From IDs to meaning

The problem with numbers

An ID is just a name tag

Text reaches a model as tokens, and every token carries an ID number. Those numbers are only labels — they were handed out by frequency, not by meaning. Token 3283 sits next to 3284 for no reason at all.