Output logits · How good is the guess?

Marking the homework

Score it by what it gave the truth

At every step a model puts a probability on each token that could come next. Once you know which token actually did come next, there’s an obvious way to mark it: how much probability did it put on the right answer? Take the negative logarithm of that and you have the loss. Certain and right scores zero; confident and wrong is punished brutally.