Proposition

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Attention is often drawn as a lookup, and the proposition below says what kind of lookup it is: a convex combination of the value vectors, with weights that are non-negative and sum to one because they came out of a softmax. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A proposition is a proved statement of moderate weight; this one is the reason the training loss needs no special treatment in the backward pass. Composing softmax with negative log-likelihood, the Jacobian of the softmax and the derivative of the logarithm cancel almost completely. ...