Demo

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A training run has a shape in time as well as in the loss curve: data preparation that nothing else can start without, a warmup whose length is set by the step size, a long stable phase, and a cooldown whose purpose is to end at a point the optimiser has settled into. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The number a language model reports while training is the mean negative log-likelihood of the token that actually came next: softmax cross-entropy over the vocabulary at every position. Its backward pass has an unusually clean closed form — the softmax Jacobian and the logarithm cancel, leaving the probabilities minus the one-hot target, divided by the number of predictions. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Step sizes in deep learning are chosen as if the loss were smooth in its parameters, and that is an assumption, not a theorem. It is the smoothness postulate: the gradient does not change arbitrarily fast, so a small step in parameter space produces a predictable change in loss. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A proof is the part of a note that cannot be skipped, and this one is short enough to show the whole pattern: write the definition, factor out the constant, use that it cancels between numerator and denominator. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A counterexample is the cheapest way to kill a plausible statement, and the one worth having ready concerns the order of matrix multiplication. The tempting assertion that \(AB = BA\) fails for almost every pair of matrices, and it fails on the smallest possible example. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The log-partition function appears wherever a normalising constant is needed, and it is trapped between two elementary bounds that are enough for most arguments: the largest term below, the largest term plus the log of the number of terms above. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Architectures are usually chosen for what they can compute, but the sharper criterion is what they cannot distinguish: matching the symmetries of the problem removes the need to learn them from data. This principle is why convolution appears in vision and why attention appears in sequences. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A criterion is a condition that decides whether an argument or a method applies, and this one is the usual substitute for convexity in the analysis of gradient descent: it bounds the loss by the size of its gradient, so a small gradient implies a small excess loss. ...