https://www.zhumengyao.com/scidraft/tags/proposition/Recent content in Proposition on SciDraft | A Hugo ThemeHugoenMon, 21 Sep 2026 10:22:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-proposition-attention-convex/Mon, 21 Sep 2026 10:22:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-proposition-attention-convex/<p><strong>Demo note.</strong> This note demonstrates what the <strong>SciDraft</strong> Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.</p> <p>Attention is often drawn as a lookup, and the proposition below says what kind of lookup it is: a convex combination of the value vectors, with weights that are non-negative and sum to one because they came out of a softmax.</p>https://www.zhumengyao.com/scidraft/notes/demo-note-proposition-gradient-cross-entropy/Mon, 21 Sep 2026 09:32:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-proposition-gradient-cross-entropy/<p><strong>Demo note.</strong> This note demonstrates what the <strong>SciDraft</strong> Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.</p> <p>A proposition is a proved statement of moderate weight; this one is the reason the training loss needs no special treatment in the backward pass. Composing softmax with negative log-likelihood, the Jacobian of the softmax and the derivative of the logarithm cancel almost completely.</p>