Probability on SciDraft | A Hugo Theme

Bayes' Rule (rule)

Mon, 21 Sep 2026 16:12:00 +0900

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.

Bayes’ rule is the rule for turning a model’s likelihood into a statement about hypotheses, and it is used twice in a language-model workflow: once to reason about uncertainty in evaluation, and once whenever a prior over prompts or parameters is combined with observed data.

Kolmogorov Axioms for a Token Distribution (axiom)

Mon, 21 Sep 2026 13:32:00 +0900

An axiom is an assumption taken as given, and the ones that hold up a language model are Kolmogorov’s three: probabilities are non-negative, the whole space has probability one, and disjoint events add. Everything else about token distributions is derived from them rather than assumed.

Log-Sum-Exp Bounds (lemma)

Mon, 21 Sep 2026 12:02:00 +0900

The log-partition function appears wherever a normalising constant is needed, and it is trapped between two elementary bounds that are enough for most arguments: the largest term below, the largest term plus the log of the number of terms above.

Cross-Entropy (definition)

Mon, 21 Sep 2026 09:02:00 +0900

The training objective of a language model has a name and a formula, and both matter when the loss curve is read: cross-entropy is the expected negative log-probability of what actually happened, measured in nats.