Demo

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A rule is a procedure applied so often that it stops looking like a theorem, and the chain rule is the extreme case: every backward pass through a stacked model is one long application of it. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. The smallest useful project around a piece of neural-network mathematics holds one idea per file and keeps the generated numbers out of the code. Nothing in it is conventional yet: a starter folder is decided by what you want to compare next, not by what a template says. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Backpropagation computes, for one layer, three gradients: one for each parameter of the layer and one to hand to the layer before it. The third is what makes deep networks trainable — without it, error could not travel past the last layer. The procedure below is the backward pass of a single dense layer. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Attention lets a model decide, for each position it is processing, which other positions to draw information from. This note states the scaled dot-product form as an algorithm rather than as an equation, so the sequence of operations — and the shape of each intermediate result — is explicit. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Attention is built on a quantity that looks like a similarity but is defined by three axioms: an inner product is symmetric, linear in each argument, and positive definite. Those three properties are what allow a score to be read as “how much one vector points along another”. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. Position information has to enter attention somewhere, and one way is to rotate the query and key vectors by a position-dependent angle. The formula is appealing because the effect on the score depends only on the difference of the two positions. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. A scaling study compares models of different sizes trained on the same token budget. If the loss falls smoothly as capacity grows, the family is in a regime where parameters still buy accuracy, and the fall can be summarised as a power law in the parameter count rather than reported as a list of separate runs. ...

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct. An algorithm is a procedure with a specification, and this one says how the attention definition is executed on a machine: scale the scores, remove the positions that are not allowed, normalise and mix. ...