Rendering A–Z

Also known as: Multiple Bibliographies, Per-Paper References, Multi-Bib Note

Shows a single note carrying more than one reference list, each drawing on its own bibliography file.

Also known as: Attention Pseudocode, Scaled Dot-Product Algorithm, Softmax Attention Procedure

Writes scaled dot-product attention as pseudocode, making the matmul, scale, mask and weighted sum explicit.

Also known as: Attention Variants, Long-Context Comparison, Attention Cost Table

Compares attention variants for long contexts by cost and memory, in the striped table style.

Also known as: Backpropagation Algorithm, Reverse-Mode Differentiation, Gradient Computation Pseudocode

States backpropagation as a two-pass procedure: a forward pass caching activations and a reverse pass accumulating gradients.

Also known as: Benchmark Survey, Third Bibliography, Benchmark References

Pairs a benchmark discussion with its own bibliography, demonstrating that bibliographies are named per note.

Also known as: Causal Attention in Python, Masked Attention Code, Attention Implementation

Implements causal attention in Python, applying a triangular mask before the softmax so no position reads ahead.

Also known as: Cross-Entropy Gradient Code, Softmax Gradient Implementation, Log-Loss Backward Pass

Computes the cross-entropy gradient through softmax in Python, showing the predicted-minus-true form in code.

Also known as: Gradient Accumulation, Micro-Batching, Accumulated Update Procedure

Shows how to reach a large effective batch size by accumulating gradients over several micro-batches before stepping.

Also known as: Layer Normalisation, LayerNorm Procedure, Per-Sample Normalisation

Gives layer normalisation as pseudocode, normalising each sample across its features and then rescaling with learned parameters.

Also known as: Masked Attention Table, Causal Mask Table, Attention Mask Reference

Tabulates how a causal mask constrains each attention position, alongside the algorithm that applies it.

Also known as: Flowchart, Flow Diagram, Process Diagram

Traces a transformer training step as a directed flowchart, showing the order of forward pass, loss and update.

Also known as: Mindmap, Concept Map, Radial Diagram

Presents the components of attention as a mindmap, grouping queries, keys, values and their relationships around a centre.

Also known as: Mermaid Overview, Diagram Gallery, Mermaid Reference

Collects the Mermaid diagram types the theme renders, each as a short working example.

Also known as: Quadrant Chart, Two-Axis Scatter, Priority Matrix

Plots attention variants on two axes so their cost and cache requirements can be compared at a glance.

Also known as: Sequence Diagram, Message Sequence Chart, Interaction Diagram

Orders the calls in an inference request as a sequence diagram, from client through the model to the token stream back.

Also known as: State Diagram, State Machine, Transition Diagram

Shows a decoding loop as a state diagram, with the transitions between prompt, generation and stop states.

Also known as: Gantt Chart, Training Schedule, Timeline Diagram

Lays a training schedule out as a Gantt chart, showing which stages run in parallel and where they wait on each other.

Also known as: Notation Reference, Symbol Table, Attention Notation

Tabulates the symbols used for multi-head attention — shapes and meanings side by side — in the academic table style.

Also known as: Power Iteration, Power Method, Dominant Eigenvector Algorithm

Computes a matrix's dominant eigenvector by repeated multiplication, the method behind spectral norm estimation.

Also known as: Softmax in Python, Stable Softmax Code, Softmax Implementation

Implements a numerically stable softmax in Python, subtracting the maximum before exponentiating.

Also known as: Chain-of-Thought Survey, Reasoning Survey, CoT Literature Review

Surveys chain-of-thought prompting, combining prose, a pseudo-algorithm, a sequence diagram and a bibliography.

Also known as: Nucleus Sampling, Top-p Sampling, Nucleus Decoding

Selects the smallest set of tokens whose cumulative probability reaches p, then samples from it rather than from the full distribution.

Also known as: Loss by Model Size, Scaling Results Table, Validation Loss Table

Sets validation loss against model size, in the bordered table style, as the tabular form of a scaling result.