Statistics

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. An identity is an equation that holds for every admissible value of its variables, and this one converts a variance into two expectations, which is often the difference between a quantity that can be estimated in one pass and one that cannot. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A corollary is what falls out of a theorem with one substitution, and the useful one here is a floor on perplexity. Since cross-entropy cannot fall below the entropy of the data, and entropy is non-negative, perplexity cannot fall below one. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A principle is a guide to design rather than a result about a particular model, and this one is the oldest in statistics: expected error splits into a term from the model being too simple, a term from it being too sensitive to the sample, and a term that no model can remove. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The exponential form of a language model’s output is not a design choice made for convenience; it is forced by an optimisation over distributions. Maximise entropy subject to matching a set of expected features, and the solution is always of the form \(\exp(\theta^{\top} f(x))\), normalised. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A conjecture is a statement believed on the weight of evidence but not proved, and the best-known one in this field concerns how loss falls as models grow. Fits are tight over the measured range, and the conjecture is that the same form continues past it. ...