Softmax on SciDraft | A Hugo Theme

Softmax of Three Scores (example)

Mon, 21 Sep 2026 13:12:00 +0900

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.

An example is where a formula stops being notation, and this one is small enough to check by hand: three scores, one softmax, three probabilities that must sum to one.

The numbers below are the ones the companion snippet prints, so the arithmetic here and the output of a few lines of code can be compared directly.

Proof of the Softmax Shift Lemma (proof)

Mon, 21 Sep 2026 12:22:00 +0900

A proof is the part of a note that cannot be skipped, and this one is short enough to show the whole pattern: write the definition, factor out the constant, use that it cancels between numerator and denominator.

Softmax Is Invariant to a Shift (lemma)

Mon, 21 Sep 2026 10:52:00 +0900

A lemma is a small result proved on the way to something larger, and this one is proved in two lines yet carries most of the numerical stability of a softmax: adding the same constant to every score changes nothing.

Temperature Cannot Sharpen (claim)

Mon, 21 Sep 2026 08:12:00 +0900

A claim is a statement put forward as true and argued for, not yet established as a theorem. The one here concerns temperature scaling: raising the temperature flattens a distribution, and it cannot make the most probable token more probable than it already was.

Numerically Stable Softmax in Python (starter)

Mon, 21 Sep 2026 07:42:00 +0900

A language model ends in a softmax: the raw scores of the last layer become a probability distribution over the vocabulary, and everything downstream — the training loss, sampling, beam search — reads that distribution rather than the scores themselves.