blogs

Notes and interactive diagrams on model architecture, memory, and training.

  • Kimi K3

    KDA and MLA attention, Attention Residuals, and Stable LatentMoE.

  • DeepSeek V4.1 Flash

    CSA2 sparse attention, shared KV memory, the causal encoder and decoder, and Engram.

  • Qwen3.8-Flash-Next

    Recurrent states, sparse attention, gated residual streams, and lexical memory in a hybrid MoE decoder.