Mermaid

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A language model writes one token at a time. The prompt is processed once, then each new token is appended to the sequence and fed back in, so generation is a loop between the model and the sampler rather than a single call. The sequence diagram below shows the participants and the messages between them. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Choosing an attention variant is a two-constraint problem rather than a ranking: the score cost decides how long a sequence can be trained on, and the per-token cache decides how long a sequence can be served, and the two orderings of the variants are not the same. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. One training step is a fixed cycle: the model predicts, the predictions are scored against the targets, the error is propagated backwards, and the parameters move a little in the direction that reduces it. The diagram below shows that cycle; the repetition is the whole of training. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A training run has a shape in time as well as in the loss curve: data preparation that nothing else can start without, a warmup whose length is set by the step size, a long stable phase, and a cooldown whose purpose is to end at a point the optimiser has settled into. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The same generation loop can be described without messages, as a machine with a small number of states. This view is useful when the interesting questions are about transitions — when decoding starts, what keeps it running, and which conditions end it. The state diagram below is the whole lifecycle of one completion. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The variants of attention that appear in long-context work are easier to keep apart by what they replace than by their names: each one gives up the full score matrix and puts something else in its place, and the something else is what the family is called. ...