Pseudocode on SciDraft | A Hugo Theme

Power Iteration with a Convergence Check (pseudocode)

Mon, 21 Sep 2026 08:39:00 +0900

Demo note. This note demonstrates what the SciDraft Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.

The largest singular value of a weight matrix is one of the numbers used to judge how a layer transforms its input, and it is usually estimated rather than computed: repeated multiplication by the matrix makes any starting vector align with the leading eigenvector, after which the Rayleigh quotient gives the value.

Layer Normalisation Written as a Function (pseudocode)

Mon, 21 Sep 2026 08:15:00 +0900

Layer normalisation rescales each token’s activations using statistics computed across the features of that token alone, which is what separates it from batch normalisation: nothing in the computation depends on the other examples in the batch.

Gradient Accumulation with an Input and Output Contract (pseudocode)

Mon, 21 Sep 2026 07:47:00 +0900

Gradient accumulation is the standard way to train with a batch larger than memory allows: several small batches are processed in turn and their gradients added before any parameter moves. The gradient of a sum is the sum of the gradients, so the result equals one step on the concatenated batch as long as the per-batch losses are averaged.