0 XP

Attention · What “it” points at

The classic test

Pronouns have to be resolved

In a sentence, each word can look at the others and pull meaning from them — that’s attention. Nowhere is it more visible than with a pronoun. “It” carries almost no meaning by itself; whatever it means has to be fetched from elsewhere in the sentence. Watch what happens when the last word changes.

The attention patterns here are hand-authored illustrations of what trained models reliably do — not weights read out of a running model. the original paper