https://www.zhumengyao.com/scidraft/tags/algorithm/Recent content in Algorithm on SciDraft | A Hugo ThemeHugoenMon, 21 Sep 2026 14:52:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-algorithm-masked-attention/Mon, 21 Sep 2026 14:52:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-algorithm-masked-attention/<p><strong>Demo note.</strong> This note demonstrates what the <strong>SciDraft</strong> Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.</p> <p>An algorithm is a procedure with a specification, and this one says how the attention definition is executed on a machine: scale the scores, remove the positions that are not allowed, normalise and mix.</p>https://www.zhumengyao.com/scidraft/notes/demo-note-algorithm-momentum/Mon, 21 Sep 2026 10:32:00 +0900https://www.zhumengyao.com/scidraft/notes/demo-note-algorithm-momentum/<p><strong>Demo note.</strong> This note demonstrates what the <strong>SciDraft</strong> Hugo theme can do for academic and scientific publishing — for research institutions, researchers and students. Its content is demonstration material only: readers should ignore whether the demo content is correct.</p> <p>The simplest optimiser with a memory is gradient descent with momentum: keep a running velocity, add the gradient to it, and step in the direction of the velocity rather than the gradient. The algorithm is short, and the behaviour it introduces is worth the two extra lines.</p>