Example

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The smallest interesting attention example has two positions and one head, which is enough for a full calculation with no matrix product larger than two by two. Working it at this size makes the two roles visible: the score decides how much the output at position two borrows from position one, and the value vectors set what is borrowed. With a causal mask, the first row has nothing to attend to but itself. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. An example is where a formula stops being notation, and this one is small enough to check by hand: three scores, one softmax, three probabilities that must sum to one. The numbers below are the ones the companion snippet prints, so the arithmetic here and the output of a few lines of code can be compared directly. ...