Statistics on SciDraft | A Hugo Theme

The Variance Identity (identity)

Mon, 21 Sep 2026 16:22:00 +0900

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.

An identity is an equation that holds for every admissible value of its variables, and this one converts a variance into two expectations, which is often the difference between a quantity that can be estimated in one pass and one that cannot.

Perplexity Is at Least One (corollary)

Mon, 21 Sep 2026 11:32:00 +0900

A corollary is what falls out of a theorem with one substitution, and the useful one here is a floor on perplexity. Since cross-entropy cannot fall below the entropy of the data, and entropy is non-negative, perplexity cannot fall below one.

The Bias-Variance Trade-Off (principle)

Mon, 21 Sep 2026 11:22:00 +0900

A principle is a guide to design rather than a result about a particular model, and this one is the oldest in statistics: expected error splits into a term from the model being too simple, a term from it being too sensitive to the sample, and a term that no model can remove.

Maximum Entropy Gives the Softmax (theorem)

Mon, 21 Sep 2026 09:22:00 +0900

The exponential form of a language model’s output is not a design choice made for convenience; it is forced by an optimisation over distributions. Maximise entropy subject to matching a set of expected features, and the solution is always of the form \(\exp(\theta^{\top} f(x))\), normalised.

Power-Law Scaling (conjecture)

Mon, 21 Sep 2026 07:22:00 +0900

A conjecture is a statement believed on the weight of evidence but not proved, and the best-known one in this field concerns how loss falls as models grow. Fits are tight over the measured range, and the conjecture is that the same form continues past it.