Demo

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The Woodbury identity rewrites the inverse of a matrix plus a low-rank correction in terms of the inverse of the original matrix and a much smaller inverse. It is the algebraic fact behind every method that updates a large system cheaply. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The linear representation hypothesis conjectures that the concepts a network has learned are encoded as directions in activation space, so that adding a concept vector to an activation moves the computation in a meaningful way. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Caching keys and values is standard practice, and the claim that justifies it is strong: incremental decoding gives exactly the same outputs as recomputing attention over the whole sequence, not an approximation of them. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Choosing an attention variant is a two-constraint problem rather than a ranking: the score cost decides how long a sequence can be trained on, and the per-token cache decides how long a sequence can be served, and the two orderings of the variants are not the same. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. One training step is a fixed cycle: the model predicts, the predictions are scored against the targets, the error is propagated backwards, and the parameters move a little in the direction that reduces it. The diagram below shows that cycle; the repetition is the whole of training. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. An axiom is an assumption taken as given, and the ones that hold up a language model are Kolmogorov’s three: probabilities are non-negative, the whole space has probability one, and disjoint events add. Everything else about token distributions is derived from them rather than assumed. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. An example is where a formula stops being notation, and this one is small enough to check by hand: three scores, one softmax, three probabilities that must sum to one. The numbers below are the ones the companion snippet prints, so the arithmetic here and the output of a few lines of code can be compared directly. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Reading a bound off a proposition is the standard use of a corollary, and the one worth having here is a ceiling on how large an attention output can be. Since the output is a convex combination of value vectors, its length can never exceed the longest of them. ...